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Executive  Summary 


An  increasing  number  of  product-development  organizations  are  “paying  attention”  to  both  ISO 
9001  and  the  Development  constellation  of  the  CMMI  Product  Suite.  This  technical  note 
compares  and  contrasts  these  two  bodies  of  knowledge  to  help  build  a  bridge  between  the  ISO 
900 1  and  CMMI  communities  that  will  promote  mutual  understanding. 

While  the  two  bodies  of  knowledge  were  developed  independently  and  for  different  purposes, 
they  have  important  connections  and  are  largely  consistent  with  each  other. 

Organizations  that  are  implementing  both  ISO  900 1  and  the  CMMI  Development  constellation 
will  benefit  from  an  understanding  of  areas  that  are  covered  fully  by  both  bodies  of  knowledge,  as 
well  as  areas  not  covered  by  both.  This  understanding  can  lead  to  efficiencies  of  operation,  as  well 
as  better  clarity  for  those  responsible  for  implementation  and  operations. 
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Abstract 


This  report  is  intended  for  organizations  and  individuals  who  have  knowledge  of  either  the 
Capability  Maturity  Model  “  Integration  (CMMI®)  Product  Suite  or  the  International  Organization 
for  Standardization  (ISO)  9000  family  of  standards,  and  are  interested  in  learning  more  about  the 
other  process  standard.  The  CMMI  for  Development,  Version  1.2  (CMMI-DEV,  VI. 2)  model  and 
the  ISO  9001:2000  standard  are  compared,  and  their  similarities  and  differences  are  noted.  This 
report  is  not  intended  to  be  an  exhaustive  or  authoritative  comparison  between  the  CMMI-DEV, 

V 1 .2  model  and  the  ISO  900 1 :2000  standard,  nor  does  it  provide  specific  guidance  for 
organizations  that  wish  to  decide  which  model  or  standard  to  adopt. 
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1  Introduction  and  Overview 


This  report  compares  and  contrasts  two  important  bodies  of  knowledge — the  International 
Standard  Organization  (ISO)  9000  family  of  quality  management  systems  standards  and  the 
Version  1.2  Capability  Maturity  Model1'  Integration  (CMMI)"  Development  constellation.  This 
introduction  provides  a  background  for  understanding  the  material  in  this  report  by  establishing 
context  and  providing  guidance  for  the  several  potential  audiences  of  this  report.  The  report’s 
primary  focus  is  on  the  primary  documents  ISO  9001:2000  and  the  CMMI  for  Development 
(CMMI-DEV),  VI  .2  model;  however,  it  is  important  to  understand  that  both  of  these  primary 
documents  are  supported  by  a  number  of  secondary  documents  and  sources  of  information. 

For  the  purposes  of  this  report,  the  term  body  of  knowledge  will  be  used  to  refer  in  a  generic  way 
to  not  only  the  principal  document  (i.e.,  ISO  9001:2000  or  CMMI-DEV,  VI. 2),  but  also  associated 
documents  and  other  artifacts.  In  the  case  of  ISO  9001 :2000,  there  are  a  number  of  related 
standards  that  not  only  supplement  the  primary  standard  (such  as  ISO  9000:2005,  ISO  9004:2000, 
and  others),  but  also  are  relevant  to  the  application  or  usage  of  ISO  9001 :2000  (such  as  ISO 
10014:2006  and  ISO  10017:2003).  As  a  result,  when  the  phrases  ISO  9001  body  of  knowledge  or 
ISO  9000 family  are  used  in  this  report,  they  refer  to  all  of  the  documents  mentioned  above,  as 
well  as  some  others.  (See  Appendices  C  and  F  for  more  details.)  Alternately,  when  the  phrase 
CMMI-DEV  body  of  knowledge  is  used  in  this  report,  it  refers  to  multiple  documents  relating  to 
CMMI-DEV,  VI. 2  and  its  application.  (See  Appendices  B  and  F  for  more  details.) 

1.1  Background 

Ever  since  the  1992  release  of  the  Capability  Maturity  Model®  (CMM)  framework  for  Software, 
many  organizations  that  produce  or  develop  products  have  a  greater  reason  to  consider  ISO  9001 
standards  as  well  as  those  of  various  other  capability  maturity  models  [Paulk  1994].  Over  time, 
the  uptake  of  these  bodies  of  knowledge  by  organizations  seeking  both  market  advantage  and 
internal  process  improvement  has  continued  to  rise.  One  recent  survey  showed  that  the  three  most 
commonly  used  technology  standards  are  ISO  9001,  CMMs,  and  ITIL,  in  that  order  [Violino 
2005], 

In  1994,  the  SEI  published  a  technical  report  that  addressed  the  relationships  between  ISO  9001 
and  the  Capability  Maturity  Model  for  Software  [Paulk  1994].  The  present  report  can  be  viewed 
as  a  successor  to  that  earlier  report,  with  a  focus  on  ISO  900 1 :2000  and  CMMI-DEV,  V 1 .2. 

1 .2  Structure  of  this  Report 

This  report  is  organized  into  four  basic  sections.  The  first  (this  section)  provides  a  brief  overview 
of  the  report’s  focus  and  organization.  The  next  two  sections  describe  the  two  bodies  of 


Carnegie  Mellon,  Capability  Maturity  Model,  and  CMMI  are  registered  in  the  U.S.  Patent  and  Trademark  Office  by 
Carnegie  Mellon  University. 
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knowledge — the  world  of  ISO  9000  and  the  world  of  CMMI-DEV.  The  final  section  provides  a 
comparative  analysis  of  the  two  bodies  of  knowledge.  Finally,  six  appendices  have  been  included 
to  provide  further  infonnation  and  facilitate  understanding,  as  well  as  to  guide  the  reader  to 
additional  important  source  materials. 

1 .3  Guidance  for  Specific  Audiences 

Individuals  already  familiar  with  one  or  the  other  of  the  two  bodies  of  knowledge  may  find  it  most 
useful  to  skip  to  the  chapter  addressing  the  area  with  which  they  are  already  familiar.  Once  this 
material  has  been  understood,  the  chapter  comparing  and  contrasting  the  two  bodies  of  knowledge 
will  be  more  meaningful  to  the  reader. 

Those  already  familiar  with  the  ISO  9000  family  might  already  speculate  as  to  how  it  relates  to 
CMMI-DEV,  and  may  have  questions  such  as  the  following: 

•  My  company  is  compliant  to  ISO  9001 — what  is  the  implication  for  the  extent  to  which  we 
satisfy  CMMI-DEV  requirements? 

•  Should  I  implement  CMMI-DEV  independently  of  ISO  9001,  or  are  there  some  areas  where 
they  overlap? 

•  Are  there  aspects  of  the  CMMI  Development  constellation  that  are  inconsistent  with  my 
implementation  of  ISO  9001? 

•  Can  my  ISO  auditor  conduct  appraisals  based  on  CMMI-DEV? 

•  Can  my  ISO  auditor  use  the  work  of  others  involved  in  appraisals  based  on  CMMI-DEV? 

Similarly,  those  already  familiar  with  the  CMMI  Development  constellation  might  already 
speculate  as  to  how  it  relates  to  ISO  9001,  and  have  questions  such  as  the  following: 

•  My  company  is  CMMI-DEV  maturity  level  3 — what  is  the  implication  for  the  extent  to 
which  we  satisfy  ISO  9001  requirements? 

•  Should  I  implement  ISO  900 1  independently  of  the  CMMI  Development  constellation  or  are 
there  some  areas  where  they  overlap? 

•  Are  there  aspects  of  ISO  9001  that  are  inconsistent  with  my  implementation  of  the  CMMI 
Development  constellation? 

•  Can  my  SCAMPI  Lead  AppraiserSM  conduct  ISO  audits? 

•  Can  my  SCAMPI  Lead  Appraiser  use  the  work  of  others  involved  in  ISO  audits? 

This  report  is  intended  to  provide  the  beginning  of  a  basis  for  addressing  these  and  other  related 
questions. 


SM  SCAMPI  Lead  Appraiser  s  a  service  mark  of  Carnegie  Mellon  University. 
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2  Overview  of  the  CMMI  Development  Constellation 


2.1  Scope  of  Application  and  Purpose 

The  CMMI  Development  constellation  provides  a  set  of  best  practices  that  are  structured  around 
the  concept  of  a  capability  maturity  model  for  organizations  that  develop  products  and  services 
and  a  set  of  appraisal  methods  and  training  courses  that  accompany  the  model.  The  CMMI  for 
Development,  V 1 .2  (CMMI -DEV,  V 1 .2)  model  is  intended  to  provide  best  practices  for 
organizations  striving  to  improve  their  product  development  capabilities.  These  best  practices 
contained  within  the  model  are  applicable  to  the  development  of  products  that  contain  one  or 
more  of  the  following  elements — hardware,  software,  firmware,  or  people. 

2.2  Background 

Preparatory  work  began  on  the  CMMI  Product  Suite  in  1997,  and  the  CMMI  Product 
Development  Team  began  operating  in  early  1998.  Their  work  culminated  in  the  release  of 
version  1.0  in  early  2001,  followed  by  the  release  of  version  1.1  in  early  2002.  The  current  version 
(1.2)  was  released  in  August  2006.  The  Vl.l  CMMI  Product  Development  Team  was  composed 
of  135  experts  from  37  organizations  representing  seven  countries;  however,  these  numbers  do 
not  include  additional  experts  or  individuals  from  other  organizations  or  countries  who 
contributed  their  expertise. 

The  CMMI  project  was  formed  to  improve  the  usability  of  Capability  Maturity  Model  (CMM) 
technology  for  a  set  of  disciplines  that  not  only  include  software  engineering,  but  also  extend 
beyond  it.  As  the  CMMI  concept  was  initially  developed,  the  scope  of  the  project  was  restricted  to 
a  few  disciplines  that  were  most  needed  by  government  and  industry. 

The  selection  of  software  engineering,  systems  engineering,  software  acquisition,  and  integrated 
product  development  CMMs  was  made  by  industry  and  government  participants  for  the  initial 
proof-of-concept  phase.  However,  the  product  suite  was  designed  to  accommodate  expansion  of 
its  discipline  coverage  as  well  as  product  and  project  life-cycle  coverage. 

In  August  2006,  version  1 .2  of  the  CMMI  Product  Suite  was  released.  Among  the  changes  was  a 
renaming  of  the  CMMI  model  to  the  CMMI  for  Development,  VI. 2  model.  This  change 
accommodated  the  anticipated  expansion  of  CMMI  practices  to  additional  areas  of  interest  (each 
new  collection  of  related  models,  appraisal  materials,  and  training  materials  for  an  area  of  interest 
was  referred  to  as  a  constellation).  The  two  new  areas  of  interest  were  acquisition  and  services; 
the  Acquisition  constellation  was  published  in  November  2007  [CMMI  Product  Team  2007]  and 
the  Services  constellation  will  be  published  in  March  2009.  Of  course,  the  CMMI-DEV,  VI. 2 
model  and  its  accompanying  appraisal  and  training  material  was  called  the  Development 
constellation.  Additional  information  about  these  other  constellations  is  available  on  the  SEI 
website. 
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2.3  Foundational  Concepts  and  Terminology 


2.3.1  Constellation 

A  CMMI  constellation  is  a  collection  of  components  that  are  used  to  construct  CMMI  models, 
training  materials,  and  appraisal  materials  in  an  area  of  interest  (e.g.,  development,  services, 
acquisition). 

2.3.2  Process  Areas 

Process  areas  (PAs)  constitute  the  primary  structural  element  of  a  CMMI  model.  A  process  area 
is  composed  of  best  practices  that,  when  implemented,  result  in  satisfaction  of  associated  goals  for 
that  process  area.  Process  areas  have  a  common  structure  that  includes  required,  expected,  and 
informative  components,  which  collectively  are  essential  to  understanding  the  intent  of  the 
process  area  and  for  proper  implementation  within  a  specific  organizational  context. 

2.3.3  Maturity  Levels 

There  are  five  maturity  levels  associated  with  the  CMMI-DEV,  V 1 .2  model,  each  of  which 
represents  a  plateau  of  organizational  capability.  In  the  case  of  the  CMMI-DEV,  V 1 .2  model,  this 
plateau  of  organizational  capability  is  relevant  for  developing  products.  These  plateaus  range  from 
maturity  level  1 — the  starting  point  of  the  maturity  scale,  representing  organizations  that  are 
primarily  ad  hoc  and  chaotic  in  their  engineering  approach — to  maturity  level  5,  which  is  the  most 
sophisticated  level  of  engineering  discipline.  An  organization  at  maturity  level  5  has  developed 
the  infrastructure  to  sustain  continuing  improvement  by  using  quantitative  and  statistical 
techniques  for  introducing  changes  in  an  orderly  and  intentional  way. 

2.3.4  Capability  Levels 

There  are  six  capability  levels  associated  with  the  continuous  representation  (see  the  definition  of 
this  term  in  Section  2.5)  of  the  CMMI-DEV,  VI. 2  model,  each  of  which  represents  a  plateau  of 
capability  associated  with  a  particular  process  area.  The  lowest  is  capability  level  0,  which  is  the 
starting  point  of  the  capability  scale.  It  represents  the  state  of  implementation  of  a  process  area 
where  one  or  more  of  the  specific  goals  of  the  process  area  are  not  yet  satisfied.  The  highest  is 
capability  level  5,  the  most  sophisticated  level  of  operation  for  a  process  area.  An  organization  at 
capability  level  5  has  the  infrastructure  to  sustain  continuing  improvement  by  using  quantitative 
and  statistical  techniques  for  introducing  changes  in  an  orderly  and  intentional  way  in  that  process 
area. 

2.3.5  Appraisal 

An  appraisal  is  the  examination  of  product  development  processes  by  a  trained  team  of 
engineering  professionals.  The  team  uses  a  process  reference  model  (such  as  the  CMMI-DEV, 

V 1 .2  model)  as  the  reference  point  for  identifying  process  strengths  and  weaknesses. 
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2.3.6  Appraisal  Team 


The  appraisal  team  is  comprised  of  professionals  who  are  trained  in  an  appraisal  methodology. 
The  team  collects  and  reviews  objective  evidence  and  uses  it  to  determine  the  extent  of  an 
organization’s  practice  implementation.  This  provides  the  basis  for  detennining  process  strengths 
and  weaknesses  and,  potentially,  goal  ratings. 

2.3.7  Appraisal  Requirements  for  CMMI  (ARC) 

The  appraisal  requirements  for  CMMI  (ARC)  define  the  set  of  requirements  applicable  to  those 
appraisal  methods  that  are  used  in  conjunction  with  CMMI  models.  The  ARC  document  was 
developed  by  the  CMMI  Product  Development  Team  and  is  considered  to  be  an  integral  part  of 
the  CMMI  Product  Suite.  See  also  “SCAMPI  Family,”  which  is  described  below. 

2.3.8  Standard  CMMI  Appraisal  Method  for  Process  Improvement  (SCAMPI) 

The  Standard  CMMI  Appraisal  Method  for  Process  Improvement  (SCAMPI)  is  the  ARC  class  A 
appraisal  method  chartered  by  the  CMMI  Steering  Group  and  developed  by  the  CMMI  Product 
Development  Team.  SCAMPI  is  intended  to  be  the  appraisal  mechanism  by  which  all  capability 
or  maturity  level  ratings  are  derived.  See  also  “SCAMPI  Family,”  which  is  described  below. 

2.3.9  SCAMPI  (A,  B,  C)  Family 

Three  appraisal  methods  constitute  the  SCAMPI  family — SCAMPI  A,  SCAMPI  B,  and  SCAMPI 
C.  Each  of  these  appraisal  methods  satisfies,  respectively,  the  ARC  criteria  for  a  Class  A,  B,  or  C 
appraisal  method.  These  three  are  designed  to  be  an  integrated  set  of  methods  that  can  be  used  as 
part  of  a  process  improvement  strategy  by  organizations  seeking  to  increase  their  product 
development  capabilities. 

2.3.10  SCAMPI  Lead  Appraiser 

A  SCAMPI  Lead  Appraiser  is  an  individual  who  has  demonstrated  the  required  skills  and 
knowledge  to  be  granted  the  authorization  to  lead  appraisals  in  the  SCAMPI  family.  The  SCAMPI 
Lead  Appraiser  is  responsible  for  ensuring  that  the  appraisal  is  planned  and  executed  in 
accordance  with  the  provisions  of  the  SCAMPI  appraisal  method. 

2.3.11  Institutionalization 

Institutionalization  is  the  ingrained  way  of  doing  business  that  an  organization  routinely  follows 
as  part  of  its  corporate  culture  and  practices.  The  process  is  ingrained  into  the  way  the  work  is 
performed  and  there  is  commitment  to  and  consistency  in  performing  the  process.  The  CMMI 
generic  practices  describe  activities  that  address  these  aspects  of  institutionalization,  and  there  is  a 
clear  progression  of  process  institutionalization  throughout  these  generic  practices. 
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2.3.12  Representation 

Representation  refers  to  the  organization,  use,  and  presentation  of  the  components  of  a  CMM. 
Two  types  of  approaches  to  presenting  best  practices  are  generally  evident:  the  staged 
representation  and  the  continuous  representation. 

2.4  Key  Product  Elements 

The  key  components  of  the  CMMI  Product  Suite  are  the  models,  appraisal  methods  (e.g.,  the 
SCAMPI  family),  and  training  courses.  These  are  addressed  in  more  detail  in  later  sections. 

2.5  CMMI  Models 

CMMI  models  are  part  of  the  CMMI  Product  Suite  and  each  CMMI  constellation.  They  are  the 
official  documents  that  contain  CMMI  best  practices  and  are  freely  available  for  download  on  the 
SEI  website.  In  addition  to  the  models  themselves,  the  SEI  website  provides  an  extensive  set  of 
supplementary  material,  including  release  notes,  errata,  and  comparisons  to  related  CMMI 
models. 

The  CMMI-DEV,  V 1 .2  model  can  be  viewed  from  two  perspectives  (called  representations) 
known  as  the  continuous  representation  and  the  staged  representation.  The  continuous 
representation  is  expressed  from  the  perspective  of  a  set  of  processes,  each  of  which  can  be 
evaluated  in  terms  of  process  capability  as  per  ISO/IEC  15504  [ISO/IEC  2003].  The  staged 
representation  is  expressed  from  the  perspective  of  the  traditional  five-stage  organizational 
maturity  scale  developed  by  the  SEI  and  first  described  in  an  SEI  technical  report  authored  by 
Watts  Humphrey  in  1987  [Humphrey  1987].  In  2007,  an  additional  model  was  released  (CMMI 
for  Acquisition,  V 1 .2),  but  this  technical  report  focuses  only  on  CMMI-DEV,  V 1 .2. 

In  addition  to  the  CMMI  for  Acquisition  model  (published  in  2006),  the  CMMI  for  Acquisition 
Primer  also  defines  effective  and  efficient  practices  for  acquisition  projects  [Richter  2008].  The 
primer  is  consistent  with  the  practices  in  the  model  and  is  a  smaller  document  that  can  be  used  by 
acquisition  organizations. 

A  model  for  service  organizations  (CMMI  for  Services)  has  also  been  developed,  and  the  initial 
version  of  this  model  was  published  in  February  2009.  CMMI  has  been  designed  to  facilitate  the 
expansion  to  additional  disciplines  over  time,  and  as  a  result,  the  community  may  expect  a 
broadening  in  the  scope  of  CMMI  models  in  the  future. 

2.6  Appraisal 

The  requirements  for  CMMI  appraisal  methods  are  established  in  the  Appraisal  Requirements  for 
CMMI,  also  known  as  the  ARC  [SCAMPI  Upgrade  Team  2006a].  The  ARC  also  establishes  three 
classes  of  CMMI  appraisal  methods:  ARC  Class  A,  ARC  Class  B,  and  ARC  Class  C. 

An  integrated  and  upwardly  compatible  family  of  CMMI  appraisal  methods  is  an  integral  part  of 
the  CMMI  Product  Suite.  These  appraisal  methods  are  suitable  for  a  wide  range  of  appraisal 
needs,  and  can  also  be  customized  based  on  the  needs  of  the  customer  organization.  The  SCAMPI 
family  of  appraisal  methods  (SCAMPI  A,  SCAMPI  B,  and  SCAMPI  C)  is  modeled  after  the 
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classification  schema  for  CMMI  appraisal  methods  defined  in  the  ARC.  The  scope  of  application 
is  broad — it  ranges  from  quick-look  style  appraisals  all  the  way  to  benchmarking  quality 
appraisals.  The  SCAMPI  family  architecture  differentiates  three  classes  of  methods  by  identifying 
the  primary  focus  of  SCAMPI  A,  B,  and  C  as  “institutionalization,”  “deployment,”  and 
“approach,”  respectively. 

The  SCAMPI  A  appraisal  method  is  defined  in  the  SCAMPI  Method  Definition  Document 
(SCAMPI  MDD),  and  is  supported  by  a  library  of  artifacts  that  are  made  available  to  SCAMPI 
Lead  Appraisers  [SCAMPI  Upgrade  Team  2006b].  The  SCAMPI  B  and  SCAMPI  C  methods  are 
defined  in  the  Handbook  for  Conducting  Standard  CMMI  Appraisal  Method  for  Process 
Improvement  (SCAMPI)  B  and  C  Appraisals  [Hayes  2005], 

The  SCAMPI  A  appraisal  method  is  the  only  member  of  the  SCAMPI  family  that  can  result  in 
capability  or  maturity  level  ratings).  SCAMPI  A  was  conceived  and  designed  to  have 
benchmarking-level  quality  and  to  become  the  standard  method  within  the  CMMI  community  for 
establishing  capability  or  maturity  level  ratings.  More  information  about  the  SCAMPI  A  appraisal 
method  is  available  on  the  SEI  website. 

SCAMPI  A  appraisals  are  led  by  an  authorized  SCAMPI  Lead  Appraiser  who  is  assisted  by 
qualified  team  members  chosen  and  trained  with  care,  mostly  from  the  evaluated  organization. 
The  examination  of  the  organization  consists  of  a  preparation  phase,  an  on-site  phase,  and  a 
reporting  phase.  The  appraisal  findings  are  presented  to  the  appraisal  sponsor  as  goal-level 
statements  that  summarize  the  gaps  in  practice  implementation.  The  SCAMPI  Lead  Appraiser 
provides  a  required  report  to  the  SEI.  Finally,  if  the  Lead  Appraiser  makes  the  request,  and  the 
appraised  organization  authorizes  the  SEI  to  do  so,  the  SEI  publishes  a  subset  of  the  appraisal 
results  on  the  published  appraisal  results  page  on  the  SEI  website. 

2.7  Training 

The  SEI  provides  a  comprehensive  set  of  training  courses  relating  to  the  CMMI  Product  Suite. 
The  prerequisites  for  these  courses  vary  for  each  course.  Some  of  these  courses  are  also  available 
from  SEI  Partners — these  are  indicated  by  asterisks  (*). 

The  CMMI  curriculum  can  be  structured  into  the  following  areas: 

•  Overview  courses 

CMMI-Based  Process  Improvement  Overview  [SEI  2009a] 

.  Model  competency 

Introduction  to  CMMI,  Version  1.2*  [SEI  2009b] 

Intermediate  Concepts  of  CMMI,  Version  1.2  [SEI  2009c] 

•  Train-the-trainer 

CMMI  Version  1 .2  Instructor  Training  [SEI  2009d] 

(also  see  appraisal  team  leader  courses  below) 

•  Process  appraisal 

SCAMPI  B  and  C  Team  Leader  Training  [SEI  2009e] 

SCAMPI  Lead  Appraiser  Training*  [SEI  20091] 
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SCAMPI  Version  1 .2  Class  A  Team  Training*  [SEI  2009g] 

SCAMPI  Version  1 .2  Class  B  Team  Training*  [SEI  2009h] 

SCAMPI  Version  1 .2  Class  C  Team  Training*  [SEI  2009i] 

Special  application  topics 

Implementing  Goal-Driven  Measurements*  [SEI  2009j] 

Analyzing  Project  Management  Indicators  [SEI  2009k] 

Advanced  topics 

Understanding  CMMI  High  Maturity  Practices  [SEI  20091] 

Improving  Process  Performance  Using  Six  Sigma*  [SEI  2009m] 

Designing  Products  and  Processes  Using  Six  Sigma*  [SEI  2009n] 

In  addition,  the  following  SEI  courses  are  directly  relevant  to  individuals  responsible  for 
process  improvement  initiatives: 

Mastering  Process  Improvement  [SEI  2009o] 

Consulting  Skills  Workshop*  [SEI  2009p] 

Managing  Technological  Change*  [SEI  2009q] 


2.8  Supporting  infrastructure 

The  supporting  infrastructure  consists  of  several  sponsoring  organizations,  a  Steering  Group 
composed  of  government,  industry,  and  SEI  members,  SEI  Partners  (organizations  licensed  by  the 
SEI  to  provide  CMMI -related  services),  and  a  steward  organization,  which  collectively  provide  a 
source  of  sustainment  and  continuing  support  for  the  adoption  and  continuing  improvement  and 
evolution  of  the  CMMI  Product  Suite. 

This  infrastructure  is  illustrated  in  Figure  1 .  For  an  explanation  of  the  acronyms  in  Figure  1 ,  see 
Appendix  D. 

The  SEI  maintains  the  following  offices: 

.  SEI  Pittsburgh  (Main  Office),  Pittsburgh,  PA,  U.S.A 

•  SEI  Colorado  Springs,  Colorado  Springs,  CO,  U.S.A. 

•  SEI  Europe,  Frankfurt,  Germany 

•  SEI  Los  Angeles,  Los  Angeles,  CA,  U.S.A 

.  SEI  Washington,  DC,  Arlington,  VA,  U.S.A 
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Figure  1:  Supporting  Infrastructure  for  CMMI 
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2.8.1  Sponsorship 


The  CMMI  Product  Suite  has  a  major  government  sponsor  (U.S.  Government)  as  well  as 
industrial  sponsorship. 

Government  sponsorship  is  provided  in  part  by  the  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  (OUSD  AT&L)  [OUSD  2008].  Industrial  sponsorship  is 
provided  by  the  Systems  Engineering  Committee  of  the  National  Defense  Industrial  Association 
(NDIA)  [NDIA  2008], 

The  government  sponsor  has  established  a  Steering  Group  to  direct  and  oversee  the  evolution  and 
maintenance  of  the  CMMI  Product  Suite  [CMMI  Steering  Group  2008],  The  group  is  composed 
of  industry  and  government  representatives,  as  well  as  the  SEI  (in  its  capacity  as  CMMI  Steward). 

2.8.2  Stewardship 

The  purpose  of  the  CMMI  Steward  is  to  ensure  the  quality  and  widespread  use  of  the  CMMI 
Product  Suite  and  to  support  its  adoption  throughout  govermnent  and  industry.  In  pursuit  of  the 
SEI’s  role  as  CMMI  Steward,  several  parts  of  the  institute  make  important  contributions. 

2.8.3  CMMI  Initiative 

The  SEI’s  CMMI  Initiative  provides  support  for  both  the  sustainment  and  evolution  of  the  CMMI 
Product  Suite.  This  initiative  includes  teams  of  professionals  that  focus  on  the  components  of  the 
product  suite. 

It  is  significant  that  the  CMMI  Initiative  is  supported  by  other  SEI  initiatives,  such  as  Software 
Engineering  Measurement  and  Analysis  (SEMA),  the  Personal  Software  Process  (PSP)  and  the 
Team  Software  Process  (TSP),  as  well  as  the  Acquisition  Support  Program  (ASP),  in  piloting  and 
measuring  the  value  and  impact  of  CMMI  adoption  in  the  community. 

2.8.4  Product  Transition  and  Development 

The  SEI’s  Product  Transition  and  Development  group  administers  the  SEI  Partner  Network, 
which,  in  turn,  provides  support  for  the  authorization  and  maintenance  of  SCAMPI  Lead 
Appraisers  and  instructors  for  the  CMMI  introductory  course. 

2.8.5  SEI  Credentials  Program 

The  SEI  offers  both  certificate  and  authorization  programs  that  pertain  to  the  CMMI  Product 
Suite. 

Certificates  are  awarded  to  course  attendees  who  complete  a  prearranged  series  of  continuing 
education  courses,  and  serve  to  recognize  successful  completion  of  an  educational  process. 
Participation  in  certificate  programs  is  a  good  way  to  build  one’s  skills  and  generally  do  not 
require  testing  or  additional  follow-up  training. 

Authorizations  often  involve  ongoing  requirements  that  must  be  met  to  keep  the  authorization 
valid.  While  SEI  authorizations  by  themselves  do  not  grant  permission  to  use  the  SEI’s 
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intellectual  property,  they  do  signify  that  an  individual  has  been  authorized  by  the  SEI  as  having 
obtained  a  specific  set  of  skills  and  knowledge  in  a  particular  area.  Authorization  allows  an 
individual  to  build  professional  credentials  through  the  objective  confirmation  of  relevant  skills. 

Certification  or  authorization  is  earned  after  an  individual  completes  a  prearranged  series  of 
courses  and  his  or  her  knowledge  is  assessed  against  a  set  of  industry-relevant  standards. 

2.9  Adoption 

The  SEI  collects  adoption  data  on  a  continuing  basis  as  part  of  its  role  as  steward  of  the  CMMI 
Product  Suite.  The  infonnation  in  the  tables  below  was  provided  by  the  SEI  and  was  current  as  of 
December  3 1 , 2008  [SEI  2009r], 

The  number  of  individuals  who  have  successfully  completed  the  indicated  CMMI-related  training 
courses  is  indicated  in  Table  1 : 

Table  1:  Individuals  who  Have  Completed  CMMI-Related  Training  Courses 


Training  Course 

Number  of 

Students 

Introduction  to  CMMI 

97,051 

Intermediate  CMMI 

2,877 

Instructor  Training  for  Introduction  to  CMMI 

678 

Understanding  CMMI  High  Maturity  Practices 

528 

The  number  of  individuals  who  have  received  authorization  credentials  from  the  SEI  is  indicated 
in  Table  2: 


Table  2:  Individuals  who  Have  Received  SEI  CMMI  Authorization  Credentials 


Authorization  Credential 

Number  of 

Students 

Introduction  to  CMMI  V 1 .2  Instructors 

439 

SCAMPI  V 1 .2  Lead  Appraiser 

497 

SCAMPI  B&C  Team  Leader 

506 

Certified  v 1. 2  High  Maturity  Lead  Appraisers 

149 
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One  important  indicator  of  the  extent  of  CMMI  adoption  is  the  number  of  SCAMPI  A  appraisals 
conducted  and  reported  to  the  SEI.  The  following  numbers  were  reported  by  the  SEI  covering  the 
period  from  the  release  of  SCAMPI  V  1.1  in  April  2002  through  December  2008  [SEI  2009r] : 

•  3,1 13  appraisals 

•  2,634  organizations 

•  1 ,882  participating  companies 

•  361  reappraised  organizations 

•  14,620  projects 

67.1%  are  organizations  outside  of  the  U.S.A. 
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3  Overview  of  the  ISO  9000  Family 


3.1  Scope  of  Application  and  Purpose 

The  ISO  9000  family  of  standards  has  been  developed  to  assist  organizations  of  all  types  and  sizes 
in  their  implementation  and  operation  of  effective  quality  management  systems.  Many  companies 
have  benefited  from  ISO  9001  implementation  when  the  scope  of  the  “organization”  was 
extended  to  the  entire  company  or  entity.  In  a  similar  fashion,  quality  management,  when  based 
on  a  systems  perspective,  represents  the  entirety  of  management’s  work  in  managing  and 
improving  performance.  Additionally,  ISO  9001  benefits  have  been  realized  when  the 
management  of  quality  has  been  extended  from  simply  product  and  process  quality  to  the 
application  of  a  system  of  processes  within  an  organization,  together  with  the  identification  and 
interactions  of  these  processes  and  their  management.  This  is  referred  to  as  the  ISO  9001 :2000 
process  approach. 

ISO  900 1  standard  is  for  quality  management  systems,  and  is  not  a 

•  product  standard 

•  quality  management  system 

•  guarantee  of  product  or  service  quality. 

3.2  Background 

The  following  is  a  brief  history  illustrating  the  origins  of  the  ISO  9000  family: 

•  Mil-Q-9858a  in  1959,  quality  standard  for  military  procurement 

•  BS  9000  in  1970,  quality  assurance  for  the  electronics  industry 

•  BS  5750  in  1979,  for  manufacturing  industries 

•  ISO  9001  in  1987,  revised  1994,  focused  on  manufactured  products 

•  ISO  900 1  in  2000,  which  added  process  approach  and  strengthened  the  areas  of  customer 
satisfaction  and  continual  improvement 

ISO  900 1  will  be  amended  in  2009  for  improved  clarity,  although  no  additional  requirements  will 
be  added. 

The  core  of  the  ISO  9000  family  contains  three  documents: 

1 .  ISO  9000:2005  Quality  management  systems — Fundamentals  and  vocabulary 

2.  ISO  9001:2000  Quality  management  systems — Requirements 

3.  ISO  9004:2000  Quality  management  systems — Guidelines  for  performance  improvements 
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3.3  Foundational  Concepts  and  Terminology 

ISO  9000  describes  the  eight  quality  management  principles  that  form  the  basis  for  the  quality 

management  system  standards  and  that  are  used  by  top  management: 

1 .  customer  focus 

2.  leadership 

3.  involvement  of  people 

4.  process  approach 

5.  system  approach  to  management 

6.  continual  improvement 

7.  factual  approach  to  decision  making 

8.  mutually  beneficial  supplier  relationships 

ISO  9001  requirements  can  be  condensed  into  five  key  statements.  The  organization  shall 

1 .  determine  the  needs  and  expectations  of  customers  and  other  interested  parties 

2.  establish  policies,  objectives  and  a  work  environment  necessary  to  motivate  the  organization 
to  satisfy  these  needs 

3.  design,  resource  and  manage  a  system  of  interconnected  processes  necessary  to  implement 
the  policy  and  attain  the  objectives 

4.  measure  and  analyze  the  adequacy,  suitability,  efficiency,  and  effectiveness  of  each  process 
in  fulfilling  its  purpose  and  objectives,  and 

5.  pursue  the  continual  improvement  of  the  system  from  an  objective  evaluation  of  its 
performance 

3.3.1  Quality  Management  System 

ISO  9000  terms  and  definitions  include  the  following: 

•  quality  management  system  (QMS):  management  system  to  direct  and  control  an 
organization  with  regard  to  quality 

•  management  system:  system  to  establish  policy  and  objectives  and  to  achieve  those 
objectives 

•  organization :  group  of  people  and  facilities  with  an  arrangement  of  responsibilities, 
authorities  and  relationships 

•  quality,  the  degree  to  which  a  set  of  inherent  characteristics  fulfils  requirements 

•  requirement,  a  need  or  expectation  that  is  stated,  generally  implied  or  obligatory 
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3.3.2  Accreditation 


ISO  accreditation  is  the  issuance  of  written  assurance  (the  certificate)  by  an  independent,  external 
body  that  has  audited  an  organization’s  management  system  and  verified  that  it  conforms  to  the 
requirements  specified  in  the  standard.  Accreditation  is  similar  to  certification  by  a  certification 
body  in  that  it  provides  a  formal  recognition  and  independent  confirmation  of  competence,  but 
accreditation  is  a  choice,  rather  than  an  obligation. 

3.3.3  Accreditation  Body 

The  accreditation  body  is  a  national  authoritative  organization  that  oversees  and  confirms  the 
competency  of  third-party  certifiers;  this  body  also  follows  the  requirements  established  by  the 
International  Accreditation  Forum.  ANSI-ASQ  National  Accreditation  Board  (ANAB)  is  the  U.S. 
accreditation  body.  ANAB  accredits  certification  bodies  (CBs)  for  ISO  900 1  quality  management 
systems. 

3.3.4  Certification 

ISO  certification  refers  to  the  issuing  of  written  assurance  (the  certificate)  by  an  independent, 
external  body  that  has  audited  an  organization’s  management  system  and  verified  that  it  confonns 
to  the  requirements  specified  in  the  standard. 

Certification  seems  to  be  the  term  most  widely  used  worldwide,  although  registration  (and 
registrar  as  an  alternative  to  certification  body)  is  often  preferred  in  North  America.  Certification 
and  registration  are  also  used  interchangeably. 

3.3.5  Certification  Body 

The  certification  body  is  an  organization  of  certified  auditors,  in  conformance  with  ISO/TR 
17021,  which  provides  an  independent  third-party  audit  of  an  organization  seeking  certification 
against  a  standard.  Note  that  certification  bodies  will  tend  to  recommend  that  companies  use  ISO 
9001  to  achieve  their  business  objectives. 

3.3.6  Registration 

ISO  registration  refers  to  the  auditing  body’s  recording  of  the  certification  in  its  client  register. 

Certification  seems  to  be  the  term  most  widely  used  worldwide,  although  registration  (and 
registrar  as  an  alternative  to  certification  body)  is  often  preferred  in  North  America.  Certification 
and  registration  are  also  used  interchangeably. 

3.3.7  Registrar 

The  registrar  is  the  same  as  the  certification  body. 

3.3.8  Auditor 

An  auditor  is  a  person  who  conducts  audits,  after  demonstrating  the  competence  to  do  so. 
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3.3.9  Audit 


An  audit  is  the  systematic,  independent,  and  documented  process  for  obtaining  audit  evidence  and 
evaluating  it  objectively  to  determine  the  extent  to  which  audit  criteria  are  fulfilled. 

3.3.10  Audit  Evidence 

Audit  evidence  consists  of  records,  statements  of  fact,  or  other  information  that  is  verifiable  and 
relevant  to  the  audit  criteria. 

3.3.11  Audit  Criteria 

Audit  criteria  consist  of  a  set  of  policies,  procedures,  or  requirements  that  are  used  as  a  reference. 

3.3.12  Competence 

Competence  is  the  demonstrated  ability  to  apply  knowledge  and  skills. 

3.3.13  Certified  Competence 

Auditor  competence  is  normally  certified  by  a  certification  body  (such  as  RABQSA  International) 
that  is  conforming  to  ISO/TR  17024.  In  this  way,  the  auditor  achieves  certified  competence. 

3.4  Key  Product  Elements 

The  key  components  of  the  ISO  9000  family  are  the  definitions,  requirements,  performance 
improvements  guidance,  and  auditing  requirements.  Audit  training  providers  are  certified  by 
national  bodies. 

3.4.1  Process  Scope 

ISO  9001  QMS  Requirements  cover  all  key  business  processes  (e.g.,  product  development  and 
delivery,  or  service  delivery)  that  affect  the  organization’s  ability  or  responsibility  to  provide  a 
product  that  meets  customer  requirements  and  all  applicable  regulatory  requirements.  ISO  9001  is 
the  most  “popular”  document  (i.e.,  has  sold  the  largest  number  of  copies)  of  the  ISO  9000  family, 
and  is  sold  by  national  bodies,  such  as  ANSI  or  ASQ  in  the  U.S  and  the  British  Standards  Institute 
(BSI)  in  the  UK.  ISO  9001  has  become  the  foundation  of  several  industry  sector  standards  such  as 
aerospace  (AS  9100),  telecom  (TL  9000),  automobile  (ISO  TS  16949),  chemical  (RC  14001), 
medical  devices  (ISO  13485),  and  petroleum  and  natural  gas  (ISO/TS  29001).  These  individual 
industry  standards  require  additional  training  and/or  experience  for  the  lead  auditors.  However, 
the  primary  focus  of  this  report  is  on  ISO  9001  and  not  on  sector  variants. 

ISO  9001  audits  (discussed  in  detail  below)  are  conducted  against  a  defined  scope  of  approval  that 
has  been  formulated  jointly  between  the  organization  and  lead  auditor.  The  scope  of  approval  is  a 
very  important  aspect  of  the  ISO  900 1  audit  and  must  clearly  identify  the  product  or  service  that 
the  organization  is  offering  for  assessment,  along  with  any  limitations  of  the  product  being 
assessed,  as  well  as  any  supporting  activities  necessary  for  producing  or  maintaining  the  product. 
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The  organizational  scope  of  sites  and  departments  to  be  included  in  the  audit  are  defined  in 
advance  of  the  audit,  as  well  as  an  audit  scope  of  the  ISO  900 1  clauses  to  be  verified  as 
implemented  by  the  organization  in  scope  of  the  audit.  Where  a  subset  of  the  organization  is  the 
organization  scope,  all  other  interfacing  parts  of  the  organization  are  treated  as  outsourced  parts  of 
the  QMS,  controlled  by  ISO  9001  clause  4.1,  and  managed  as  suppliers  under  the  ISO  9001  clause 
7.4.  The  scope  of  the  audit  is  controlled  by  ISO  9001  clause  1.2  that  defines  what  parts  of  the  ISO 
9001  clauses  may  be  “excluded”  from  the  audit  scope.  If  a  requirement  described  by  an  ISO  9001 
clause  is  actually  performed  within  the  scope  of  the  organization  or  an  outsourced  entity,  it  must 
be  included  in  the  audit  scope.  The  organization  scope  and  the  audit  scope  are  the  basis  for  the 
Registrar’s  quote  for  services  and  how  many  auditors  are  assigned  to  the  audit. 

These  rules  are  intended  to  ensure  that  the  final  certificate  of  approval  does  not  mislead  potential 
stakeholders;  for  example,  customers  of  the  organization.  It  must  be  noted,  however,  that 
activities  that  are  an  inherent  part  of  the  organization’s  line  of  business  or  that  are  required  by  the 
ISO  900 1  standard  cannot  be  excluded  from  the  scope  of  approval. 

In  addition  to  the  scope  of  approval,  the  lead  auditor  will  also  need  to  establish  the  locations  at 
which  these  activities  are  either  undertaken  or  are  supported  within  the  organization.  Once  certain 
other  requirements,  such  as  the  size  of  the  organization,  geographic  aspects,  or  working  patterns 
have  been  identified,  the  lead  auditor  will  be  in  a  position  to  prepare  a  plan  for  the  audit. 

In  practice,  organizations  will  define  the  product  or  service,  the  locations  involved,  and  the 
limitations  that  are  subject  to  the  scope  of  approval  (or  certification).  Apart  from  clauses 
identified  within  Section  7  of  the  ISO  900 1  standard,  all  clauses  are  applicable,  and  the 
organization  under  certification  must  demonstrate  adherence  to  their  specifications  in  relation  to 
the  defined  scope  of  approval. 

The  clauses  in  Section  7  can  be  deemed  not  applicable,  but  the  rationale  for  this  judgment  within 
the  organization  scope  must  be  defined  as  an  exclusion  (per  clause  1.2  in  the  organization’s 
Quality  Manual),  and  agreed  upon  by  the  certification  body.  As  described  above,  clause 
requirements  are  sometimes  fulfilled  by  a  subset  of  the  organization  outside  the  audited 
organization  scope.  These  other  interfacing  parts  of  the  organization  (or  external  contractors)  are 
treated  as  outsourced  parts  of  the  QMS,  controlled  by  ISO  9001  clause  4.1,  and  managed  as 
suppliers  under  the  ISO  9001  clause  7.4. 


3.4.2  Sector-Specific  Variants 

There  are  a  number  of  sector-specific  variants  and  guides  associated  with  ISO  9001,  with  the  Tick 
IT  guide  probably  being  one  of  those  that  relate  most  closely  with  the  CMM  for  Software 
(previously)  and  now  with  CMMI-DEV,  VI. 2.  Associated  standards  and  guides  include  ISO 
9000-3  (now  superseded  by  ISO  90003),  which  provides  interpretation  of  the  ISO  9001 
requirements  for  the  IT  sector.  It  must  be  noted,  however,  that  this  is  not  an  assessable  standard. 
ISO  90003  assists  stakeholders,  such  as  quality  managers  or  auditors,  in  translating  the  generic 
terms  used  in  ISO  9001  into  terms  that  are  commonly  understood  in  the  IT  industry. 
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The  Tick  IT  guide  elaborates,  in  a  detailed  and  informative  manner,  on  ISO  90003 ’s  more  general 
translation  of  ISO  9001  into  terms  commonly  understood  in  the  IT  industry.  Supporting  the  Tick 
IT  guide  is  the  Tick  IT  scheme,  which  is  currently  recognized  by  the  UK  and  Swedish 
accreditation  bodies.  Organizations  can  adopt  the  Tick  IT  scheme  and  will,  if  judged  acceptable 
by  the  Tick  IT  auditor,  be  awarded  an  accredited  certificate  bearing  the  Tick  IT  logo  in  addition  to 
the  accreditation  body  logo.  The  Tick  IT  scheme  principally  requires  that  organizations,  primarily 
in  the  IT  sector,  implement  and  demonstrate  a  quality  management  system  that  satisfies  the  ISO 
9001  requirements,  as  guided  by  the  Tick  IT  guide  and  audited  by  Tick  IT  Lead  Auditors. 

The  Tick  IT  guide  is  a  required  input  to  the  auditing  and  certification  process  when  a  Tick  IT 
certification  is  being  sought.  As  with  all  other  ISO  900 1  -based  industry  sector  schemes,  an 
organization  may  achieve  ISO  9001  certification — but  not  the  sector  scheme’s  certification — if 
the  sector  scheme’s  required  elaborations  on  ISO  9001  have  not  been  demonstrated  in  the  audit. 
The  interpretation  and  guidance  is  intended  to  help  IT  organizations  understand  and  implement 
processes,  procedures,  and  practices  that  satisfy  the  ISO  900 1  requirements.  As  a  simple  example, 
clause  7.5.1  of  ISO  9001:2000  requires  that  Product  Identification  and  Traceability  be 
implemented  and  demonstrated,  and  the  Tick  IT  guide  helps  to  explain  how  Configuration 
Management  would  address  this  ISO  900 1  requirement. 

Tick  IT  is  a  scheme  for  certification  in  the  Information  Technology  sector  sponsored  by  the  UK 
government.  A  Certification  Body  must  be  accredited  by  UKAS  to  certify  this  scheme. 

The  current  list  of  sector-specific  documents  can  be  found  in  TC176  N881.  [ISO  2006b]  A  list  of 
registrars  can  be  found  at  http://www.iso.org/iso/en/info/ISODirectory/countries.html. 

3.5  Appraisal 

In  ISO  900 1 ,  an  appraisal  is  called  an  audit  (see  the  above  definition).  The  audit  is  conducted 
using  ISO  1901 1,  or  QE1901  IS  within  the  U.S.,  for  internal  quality  auditors.  Conformance  to  ISO 
9001  is  rated  as  either  pass  or  fail.  While  ISO  9001  has  a  maturity  scheme  in  ISO  9004  Appendix 
A,  it  is  informative  and  does  not  affect  ISO  9001  registration  audits.  ISO  9001  audit  results  are 
recorded  in  the  final  audit  report  for  registration,  and  findings  that  affect  registration  are  recorded 
as  non-conformances,  typically  as  either  minor  or  major. 

In  the  case  of  a  minor  non-conformance  (in  the  spirit  of  continual  improvement)  and  following  an 
exchange  of  correspondence  concerning  the  corrective  action  planned  or  taken  by  the  audited 
organization,  a  certificate  indicating  certification  is  issued,  the  audited  company  is  registered  as 
conforming  to  ISO  900 1  standards,  and  the  actual  corrective  action  taken  is  verified  as  effective  at 
the  next  surveillance  audit  of  the  audited  organization.  This  may  occur  as  rapidly  as  the  audited 
organization  can  respond  to  the  minor  audit  findings,  and  the  Lead  Auditor  confirms  the 
suitability  of  the  action  recorded  in  the  correspondence  from  the  audited  organization. 

In  the  case  of  a  major  non-conformance  (once  again,  in  the  spirit  of  continual  improvement) 
following  an  exchange  of  correspondence  on  the  corrective  action  that  is  actually  taken  by  the 
audited  organization,  a  follow-up  audit  on  only  the  major  non-conformance  is  performed  to  verify 
effective  implementation.  Once  effective  implementation  is  verified,  a  certificate  indicating 
certification  is  issued,  and  the  company  is  registered  as  conforming  to  ISO  9001  standards. 
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ISO  9001  audits  can  be  conducted  internally,  as  required  by  clause  7.4.1;  by  a  second  party,  such 
as  the  organization’s  customer,  for  the  purpose  of  contract  conformance;  or  by  an  independent 
third  party,  called  a  certification  body,  for  the  purpose  of  achieving  a  certification. 

In  ISO  900 1  tenns,  organizations  undergo  audits  conducted  by  one  or  more  auditors  led  by  a  lead 
auditor.  The  audit  process  consists  of  a  number  of  activities,  some  mandatory  and  some  optional, 
but  almost  always  in  a  defined  order.  The  first  activity,  which  is  optional,  might  be  for  the 
organization  to  request  a  preliminary >  audit,  which  is  conducted  by  the  lead  auditor.  The 
preliminary  audit  includes  a  document  review  (called  the  adequacy  audit)  to  ensure  that  the 
organization  has  an  adequate  framework  against  which  an  audit  can  be  conducted.  A  preliminary 
audit  is  used  by  the  organization  to  get  a  feel  for  the  areas  of  weakness  or  concern  prior  to  any 
formal  audit  activities.  The  approach  may  range  from  conducting  a  broad  high-level  check  of  the 
processes  across  a  wide  range  of  business  areas  to  a  very  detailed  check  on  activities  undertaken 
in  specific  parts  of  the  business.  The  result  of  the  preliminary  audit  is  usually  a  report  by  the  lead 
auditor  that  highlights  any  aspects  that  might  cause  a  problem  which  would  prevent  certification 
during  a  formal  certification  audit. 

The  next  stage  of  the  audit  process,  which  is  mandatory,  is  to  conduct  the  stage  1  audit,  which  is 
better  known  as  the  documentation  review  [ISO  2004].  This  audit  is  intended  primarily  for 
scoping  and  planning  a  certification  audit  (the  stage  2  audit)  and  to  allow  the  auditor  to  obtain  a 
more  thorough  understanding  of  the  organization.  The  auditor  examines  the  quality  management 
system  documentation  to  ensure  that  the  organization  has  defined  processes  and  procedures  that 
will  address  the  requirements  of  the  ISO  900 1  standard.  The  rationale  for  this  audit  is  that  if  the 
organization’s  defined  system  does  not  address  the  requirements,  there  is  a  high  risk  that  its 
practices  are  insufficient  as  well.  This  won’t  always  be  the  case,  but  since  defined  processes  are 
required,  it  can  be  more  cost  effective  to  start  with  those  than  to  conduct  a  full  audit  to  determine 
that  the  practice  is  deficient  because  defined  processes  are  absent. 

In  addition  to  examining  the  defined  management  system,  the  lead  auditor  will  check  the 
implementation  of  some  aspects  of  the  standard,  specifically  that  management  commitment  is 
present  and  that  the  plan-do-check-act  cycle  is  either  well  underway  or  has  iterated  at  least  once. 
One  of  the  fundamental  aspects  of  the  ISO  9001 :2000  standard  is  the  concept  of  plan-do-check- 
act;  in  order  to  successfully  achieve  ISO  9001 :2000  certification  the  organization  must 
demonstrate  its  following  of  this  cycle.  In  addition  to  checking  process  documentation,  the  lead 
auditor  schedules  interviews  with  senior  management  to  check  management  commitment, 
investigate  management  reviews,  examine  internal  audit  records,  and  understand  process 
measures  and  analysis. 

The  result  of  the  stage  1  audit  is  a  report  that  defines  areas  of  concern,  which  are  usually 
expressed  as  deficient  areas,  including  comments,  and  observations.  The  lead  auditor  usually  also 
indicates  whether  the  defined  time  scales  for  conducting  the  main  (stage  2)  audit  are  practical  and 
appropriate. 

Usually  within  three  to  six  months,  and  assuming  that  no  major  issues  or  concerns  were  raised  at 
the  stage  1  audit,  the  stage  2  compliance  audit  is  undertaken.  This  is  the  main  mandatory  part  of 
the  audit,  in  which  evidence  is  gathered  and  consolidated  to  demonstrate  that  the  defined  and 
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planned  arrangements  are  being  effectively  implemented.  The  schedule  usually  involves  an 
opening  meeting,  audit  activities,  a  team  meeting,  and  a  closing  meeting.  The  most  popular  audit 
approach  is  to  conduct  interviews  with  key  members  of  staff  and,  where  appropriate,  to  witness 
activities  being  performed.  Additionally,  evidence  presented  at  the  time  of  the  interviews  is 
examined  to  confirm  the  statements  that  were  made. 

The  result  of  a  stage  2  audit  is  usually  a  report  that  contains  details  of  the  audit,  including 
comments,  observations,  and  non-conformances.  Additionally,  if  the  organization  is  compliant 
with  the  requirements  of  ISO  900 1 :2000,  a  recommendation  for  approval  will  also  be  made.  In 
most  cases,  a  recommendation  for  approval  will  only  be  made  when  there  are  zero  non¬ 
conformances. 

The  certification  body  conducts  a  technical  review  of  the  recommendation,  and  if  it  is  deemed 
acceptable,  a  certificate  of  approval  will  be  raised.  The  certificate  identifies  the  details  of  the 
organization  being  audited,  the  scope  of  approval,  the  locations  where  the  scope  of  approval 
applies,  and  the  certificate’s  expiration  date  (which  is  usually  three  years  from  the  issue  date). 

Once  successful,  the  organization  is  required  to  undergo  a  series  of  surveillance  visits  over  the 
certification  period — usually,  a  visit  by  an  auditor  every  six  months  for  three  years,  after  which 
time,  a  recertification  is  performed.  There  is  an  accepted  variance  to  this  for  small  firms,  wherein 
the  surveillance  period  is  extended  to  nine  months  in  the  same  certification  period.  These 
surveillance  visits  do  not  involve  re-examination  of  entire  system  but  only  of  sample  elements,  in 
order  to  provide  some  degree  of  confidence  that  the  system  continues  to  be  implemented 
effectively.  However,  certain  key  aspects  are  almost  always  checked,  including  management 
commitment,  system  changes,  management  reviews,  internal  audits,  handling  customer 
complaints,  corrective  and  preventative  actions,  measurements,  and  continuous  improvements. 

The  last  surveillance  visit  is  replaced  by  a  recertification  visit,  during  which  the  whole  system  is 
rechecked.  If  this  recheck  indicates  that  the  organization  remains  in  compliance,  the  certificate  is 
extended  for  an  additional  three  years. 

Throughout  the  certification  period,  an  organization  can  request  changes  to  approval  to  extend  the 
scope  of  certification.  The  certification  body  analyzes  such  requests  and  ensures  that  adequately 
skilled  auditors  and  audit  time  are  included  in  the  surveillance  visit,  or  that  an  additional  visit  is 
included  to  address  the  request.  The  same  approach  used  in  the  initial  audit  is  implemented  on  a 
much  smaller  scale  to  address  the  change  to  approval.  If  the  audit  is  successful,  the  certificate  will 
be  updated  and  reissued,  although  the  three-year  certification  period  will  not  be  extended. 
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3.6  Training 

Training  providers  are  third  parties  who  are  certified  by  bodies  such  as  RABQSA  International, 
which  is  accredited  by  the  International  Standard  for  Personnel  Certification,  ISO/IEC 
17024:2003.  RABQSA  has  the  ability  to  design,  develop,  and  manage  personnel  certification 
schemes.  Its  courses  are  used  by  over  50  training  companies,  and  its  ISO  9001  specific  courses 
are 

•  RABQSA -AU — Auditing  Management  Systems 

•  RABQSA -TL — Leading  MS  Audit  Teams 

.  RABQSA-QM — Quality  Management  Systems 

The  audiences  for  these  courses  are 

•  provisional  auditors 

•  auditors 

•  principal  auditors 

•  lead  auditors 

•  business  improvement  auditors 

Another  certification  body  is  the  International  Register  of  Certified  Auditors  (IRCA)  in  the  UK. 
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3.7  Supporting  infrastructure 


Figure  2:  Supporting  Infrastructure  for  ISO  9001 


3.8  Adoption 

By  the  end  of  2005,  776,608  ISO  9001  certificates  had  been  issued  in  161  countries  and 
economies,  and  many  more  companies  have  self-declared  conformance  to  ISO  900 1  standards. 
Software  engineering  companies  have  used  ISO/IEC  90003  for  guidance  on  applying  software  to 
ISO  9001. 
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4  Comparative  Analysis 


In  this  section,  we  examine  the  differences  between  the  ISO  9000  framework  and  the  CMMI- 
DEV,  V 1 .2  model.  The  purpose  is  not  to  suggest  that  one  approach  is  superior  to  another,  but  to 
help  readers  understand  and  appreciate  each  model’s  differences,  and  to  see  how  these  differences 
are  driven  by  both  the  original  and  the  evolved  purposes  of  each  model  or  framework.  Since  many 
readers  of  this  report  “live”  in  one  or  the  other  bodies  of  knowledge,  we  hope  that  discussing  the 
bodies  of  knowledge  in  this  way  will  facilitate  a  deeper  understanding  and  appreciation  for  the 
other. 

4.1  Terminology 

Appendix  E  identifies  important  terms  that  the  two  models  have  in  common,  but  for  which  the 
terms  or  definitions  are  not  identical. 

The  foremost  of  these  differences  between  the  two  models  are  the  fundamental  evaluation  criteria, 
which  are  suggested  by  the  use  of  the  terms  “audit”  and  “appraisal.”  As  it  happens,  these  terms  do 
connote  some  important  differences  between  the  two  approaches. 

The  above  terms  are  closely  associated  with  the  notions  of  “certification”  and  “authorization.” 

The  term  certification  is  used  in  the  ISO  9000  family  to  mean  that  a  credential  (the  certificate)  has 
been  awarded  to  the  organization,  and  that  both  the  certification  body  and  the  auditor  certify  that 
the  audit  report  documents  the  scope  of  the  organization  and  the  scope  of  the  audit,  as  well  as 
identifying  any  non-conformances. 

The  Registrar  records  subsequent  client  corrective  actions  that  are  accepted  by  the  Registrar,  and 
notes  the  basis  for  the  decision  (which  is  only  within  the  process  bounds  reflected  by  the  audit 
evidence  sampled  as  recorded  in  the  auditor's  audit  notes  on  file  with  the  Registrar). 

The  ISO  scheme  also  provides  for  certification  of  individuals.  Figure  2  in  Section  2.8  shows  that 
the  certification  body  for  auditors  certifies  ISO  900 1  auditors. 

In  CMMI,  the  single  closest  equivalent  to  the  ISO  900 1  certification  certificate  is  the  SEI 
Appraisal  Disclosure  Statement  (ADS) — a  required  part  of  the  SCAMPI  Lead  Appraiser’s  report 
to  the  SEI — that  is  reviewed  and  approved  by  the  SEI.  This  is  a  similar  process  to  ISO  900 1 , 
where  the  Registrar  Certification  Department  approves  the  audit  report,  client  response  to  non¬ 
conformances  and  auditor  acceptance,  the  audit  scope,  and  the  certificate.  Also,  this  department 
typically  records  the  client’s  Registration  in  the  Registrar's  Directory. 

In  the  CMMI  world,  there  is  no  single  equivalent  to  certification.  One  would  say  that  an 
organization  has  achieved  a  designated  maturity  level  and  that  after  following  an  internal  quality 
review  process,  the  SEI  has  accepted  the  appraisal  as  valid.  The  term  authorization  is  sometimes 
confused  with  certification  by  those  impacted  by  CMMI -based  improvement — in  fact,  the  two 
terms  are  only  loosely  associated  with  one  another.  Authorization  has  a  well-defined  meaning  in 
the  CMMI  world — it  means  that  an  individual  has  successfully  fulfilled  SEI  requirements  related 
to  a  subject  matter  area  and  has  demonstrated  the  skills  requisite  to  being  granted  permission  by 
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the  SEI  to  deliver  a  particular  service  on  behalf  of  an  SEI  Partner.  For  example,  an  individual  who 
has  been  authorized  by  the  SEI  as  a  SCAMPI  Lead  Appraiser  or  a  High  Maturity  Lead  Appraiser 
is  allowed  to  use  SEI  materials  to  lead  a  SCAMPI  A  appraisal,  as  long  as  it  is  conducted  under  the 
auspices  of  an  SEI  Partner  that  has  a  license  for  that  service. 

4.2  Key  Product  Elements 

As  an  international  standard,  ISO  900 1  is  composed  of  several  kinds  of  informational  elements 
that  are  broadly  classified  into  two  types — normative  and  informative.  For  the  purposes  of 
comparison  with  CMMI-DEV,  the  normative  element  called  requirement  is  closest  in  meaning  to 
the  “required”  category  of  information  in  CMMI-DEV,  VI. 2 — that  is,  goal  statements.  In  ISO 
900 1 ,  requirements  are  expressions  in  the  standard  conveying  criteria  to  be  fulfilled  if  compliance 
with  the  standard  is  to  be  claimed  and  from  which  no  deviation  is  permitted.  There  are  strict  style 
rales  for  the  expression  of  ISO  requirements —  they  contain  the  word  “shall”  or  the  phrase  “shall 
not.” 

Similarly,  the  normative  element  called  recommendation  is  closest  in  meaning  to  the  “expected” 
category  of  information  in  CMMI-DEV,  V 1 .2 — which  comprises  practice  statements.  In  ISO 
900 1 ,  a  recommendation  is  an  expression  conveying  that  among  several  possibilities  one  is 
regarded  as  particularly  suitable,  without  mentioning  or  excluding  others;  or  that  a  certain  course 
of  action  is  preferred  (but  not  necessarily  required),  or  that  (in  the  negative  form)  a  certain 
possibility  or  course  of  action  is  deprecated,  but  not  prohibited.  A  recommendation  would  contain 
the  words  “should”  or  “should  not.” 

Finally,  all  other  informational  elements  in  ISO  9001  would  correspond  to  what  is  referred  to  in 
CMMI  as  informative  material. 

ISO  9001,  like  CMMI,  has  a  maturity  scheme  in  Appendix  A  of  ISO  9004;  however,  it  is 
guidance  material  and  it  has  rarely  been  used  in  practice.  The  maturity  scheme  in  CMMI  plays  a 
more  central  role  to  the  CMMI  community  than  the  ISO  9001  maturity  scheme  does  to  the  ISO 
9001  community. 

4.3  Process  scope 

4.3.1  Organizational  Scope 

ISO  900 1  functions  as  a  generic  standard  for  all  organizations  and  thus  is  written  at  a  high  level 
(only  1 9  pages).  The  scope  of  ISO  900 1  is  broader  than  that  of  CMMI-DEV,  V 1 .2,  as  it  may  be 
applied  to  part  of  an  organization,  or  to  all  aspects  of  the  organization  that  are,  in  principle, 
encompassed  by  the  provisions  of  ISO  9001. 
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4.3.2  Level  1  Coverage  Comparison 


The  following  table  shows  a  summary  comparison  of  the  content  relationships  between  the  major 
clauses  of  ISO  900 1  and  the  CMMI-DEV,  V 1 .2  process  categories.  A  green  cell  indicates  a 
significant  or  notable  amount  of  overlap,  while  a  grey  cell  indicates  minimal  or  no  content 
overlap.  We  call  this  a  level  1  mapping  to  indicate  that  it  reflects  high-level  relationships  that  are 
relative  to  the  fundamental  architectural  components  of  the  two  bodies  of  knowledge. 


Table  3:  Summary  Comparison  of  the  Content  Relationships  Between  the  Major  Clauses  of  ISO  9001 
and  the  CMMI-DEV,  VI. 2  Process  Categories 


Quality 

Management 

System  (4) 

Management 
Responsibility  (5) 

Resource 

Management  (6) 

Product  Realization 
(7) 

Measurement, 

Analysis  and 
Improvement  (8) 

Process 

management 

Project 

management 

Engineering 

Support 
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The  following  table  shows  a  more  detailed  comparison  of  how  the  generic  practices  overlap  with 
major  clauses  of  ISO  9001.  A  green  cell  means  that  the  relevant  generic  practice  is  adequately 
covered  by  the  provisions  of  the  associated  clauses,  while  a  grey  cell  indicates  that  the  generic 
practice  is  not  adequately  addressed  by  any  of  the  clauses  in  the  referenced  section  of  ISO  9001. 


Table  4:  Detailed  Comparison  of  how  Major  Clauses  of  ISO  9001  Overlap  with  the  CMMI-DEV,  VI. 2 
Generic  Practices 


ISO  9001 :2000  Sections 

CMMI-DEV,  VI  .2 

Generic  Practices 

Quality 

Management 

System  (4) 

Management 
Responsibility  (5) 

Resource 
Management  (6) 

Product  Realization 
(7) 

Measurement, 
Analysis  and 
Improvement  (8) _ 

Perform  Specific  Practices  (1.1) 

Establish  an  Oraanizational  Policv  (2.1) 

Plan  the  Process  (2.2) 

Provide  Resources  (2.3) 

Assian  ResDonsibilitv  (2.4) 

Train  PeoDle  (2.5) 

Manaae  Confiaurations  (2.6) 

Identify  and  Involve  Relevant  Stakeholders  (2.7) 

Monitor  and  Control  the  Process  (2.8) 

Obiectivelv  Evaluate  Adherence  (2.9) 

Review  Status  with  Hiaher  Level  Manaaement  (2.10) 

Establish  a  Defined  Process  (3.1) 

Collect  ImDrovement  Information  (3.2) 

Establish  Quantitative  Obiectives  for  the  Process  (4.1) 

Stabilize  SubDrocess  Performance  (4.2) 

Ensure  Continuous  Process  ImDrovement  (  5.1) 

Correct  Root  Causes  of  Problems  (5.2) 
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4.3.3  Mid-Level  Coverage  Comparison  (CMMI-DEV,  VI. 2  to  ISO  9001) 


The  following  table  shows  a  mid-level  comparison  of  the  content  relationships  between  CMMI- 
DEV,  VI. 2  and  ISO  9001. 

A  blue  cell  indicates  that  the  requirements  of  the  ISO  9001  standard  would  be  considered  to  be 
satisfied  by  CMMI-DEV,  V 1 .2  practices  (within  the  relevant  process  area)  without  any  significant 
guidance  or  interpretation. 

A  green  cell  indicates  that  the  requirements  of  the  ISO  9001  standard  would  be  satisfied  if 
CMMI-DEV,  V 1 .2  practices  (within  the  relevant  process  area)  were  fully  interpreted  and 
implemented.  For  example,  clause  4.2.4  discusses  the  requirement  for  quality  records,  but  CMMI- 
DEV,  V 1 .2  does  not  have  a  process  area  that  is  directly  equivalent  to  quality  records.  However, 
generic  practice  3.2  of  the  CMMI-DEV,  V 1 .2  model  does  require  data,  information,  or 
measurements  to  be  retained  throughout  the  model  process.  A  yellow  cell  indicates  that  the 
requirements  of  the  ISO  900 1  standard  may  be  satisfied  by  CMMI-DEV,  V 1 .2  practices  (within 
the  relevant  process  area)  if  significant  additional  interpretation  is  undertaken.  For  example,  up  to 
maturity  level  3,  there  is  no  direct  requirement  to  conduct  corrective  action  in  a  manner  that 
involves  root  cause  analysis  and,  therefore,  the  requirements  of  the  CMMI-DEV,  V 1 .2  model 
would  be  satisfied  with  simple  remedial  action  until  Causal  Analysis  and  Resolution  is 
implemented  for  the  organization  to  achieve  maturity  level  5.  However,  should  an  organization 
perform  some  form  of  root  cause  analysis  on  issues — in  effect,  start  to  address  maturity  level  5 
needs — then  this  would  satisfy  clause  8.5.1  of  ISO  9001:2000. 

A  grey  cell  indicates  that  these  requirements  would  be  satisfied  through  implementation  of  other 
ISO  9001  clauses  and  therefore  are  not  addressed  separately.  For  example,  clause  8.5.1,  which 
requires  continual  improvement,  is  actually  demonstrated  through  a  number  of  other  clauses: 
quality  policy  (5.3)  drives  objectives  (5.4.1),  which  are  checked  through  internal  audit  (8.2.2)  and 
analysis  of  data  (8.4),  leading  to  corrective  and  preventive  action  (8.5.2A3),  which  is  reviewed  by 
management  (5.6).  Finally,  a  white  cell  indicates  that  there  is  no  significant  overlap  between  the 
CMMI-DEV,  V 1 .2  model  and  the  ISO  900 1  framework  in  these  particular  areas. 
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Table  5:  Mid-Level  Comparison  of  the  Content  Relationships  Between  CMMI-DEV,  VI. 2  and  ISO 
9001 


ISO  9001 :2000  Clauses 

Level  2 

Level  3 

Generic  Practices 

2 

O 

cr 

cl 

0. 

O 

2 

2 

2 

< 

o 

2 

O 

a 

cc 

C/3 

CL 

cr 

< 

CL 

O 

a 

CL 

o 

o 

2 

2 

2 

cr 

O 

2 

O 

o 

o 

cr 

o 

CL 

O 

CL 

C3 

CL 

o 

CL 

C D 

<3 

0 

CL 

o 

CL 

o 

CL 

o 

CL 

o 

CL 

O 

CL 

O 

CL 

0 

CL 

0 

<3 

CL 

C3 

4 

Quality  management  system 

4.1 

General  requirements 

4.2 

Documentation  requirements 

4.2.1 

General 

4.2.2 

Quality  manual 

4.2.3 

Control  of  documents 

4.2.4 

Control  of  records 

5 

Management  responsibility 

5.1 

Management  commitment 

5.2 

Customer  focus 

5.3 

Quality  policy 

5.4 

Planning 

5  4.1 

Quality  objectives 

5.4.2 

Quality  management  system  planninq 

5.5 

Responsibility,  authority  and  communication 

5.5.1 

Responsibility  and  authority 

5.5.2 

Management  representative 

5.5.3 

Internal  communication 

5.6 

Management  review 

5.6.1 

General 

F 

5.6.2 

Review  input 

5.6.3 

Review  output 

6 

Resource  management 

6.1 

Provision  of  resources 

6.2 

Human  resources 

6.2.1 

General 

6.2.2 

Competence,  awareness  and  training 

6.3 

Infrastructure 

6.4 

Work  environment 

7 

Product  realization 

7.1 

Planninq  of  product  realization 

7.2 

Customer-related  processes 

7.2.1 

Determination  of  requirements  related  to  the  product 

7.2.2 

Review  of  requirements  related  to  the  product 

7.2.3 

Customer  communication 

7.3 

Design  and  development 

7.3.1 

Design  and  development  planning 

7.3.2 

Design  and  development  inputs 

7.3.3 

Desiqn  and  development  outputs 

7.3.4 

Design  and  development  review 

7.3.5 

Design  and  development  verification 

7.3.6 

Design  and  development  validation 

7.3.7 

Control  of  design  and  development  changes 

7.4 

Purchasing 

7.4.1 

Purchasing  process 

7.4.2 

Purchasing  information 

7.4.3 

Verification  of  purchased  product 

7.5 

Production  and  service  provision 

7.5.1 

Control  of  production  and  service  provision 

— 

=F 

7.5.2 

Validation  of  processes  for  production  and  service  provision 

7.5.3 

Identification  and  traceability 

7.5.4 

Customer  property 

7.5.5 

Preservation  of  product 

7.6 

Control  of  monitoring  and  measuring  devices 

8 

Measurement,  analysis  and  improvement 

8.1 

General 

8.2 

Monitorinq  and  measurement 

8.2.1 

Customer  satisfaction 

8.2.2 

Internal  audit 

8.2.3 

Monitorinq  and  measurement  of  processes 

8.2.4 

Monitoring  and  measurement  of  product 

8.3 

Control  of  nonconforming  product 

8.4 

Analysis  of  data 

8.5 

Improvement 

8.5.1 

Continual  improvement 

8.5.2 

Corrective  action 

8.5.3 

Preventive  action 
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4.3.4  Mid-Level  Coverage  Comparison  (ISO  9001  to  CMMI-DEV,  VI. 2) 


In  general,  ISO  900 1  does  not  directly  cross-reference  with  CMMI-DEV,  V 1 .2  process  areas, 
such  as  Risk  Management,  Decision  Analysis  and  Resolution,  Organizational  Process 
Performance  (which  is  partially  met  below).  Product  Integration,  or  Quantitative  Project 
Management.  In  general,  all  other  CMMI-DEV,  V 1 .2  process  areas  are  supported  to  varying 
extents  by  ISO  900 1  clauses,  though  no  single  CMMI-DEV,  V 1 .2  process  area  is  fully  supported 
by  ISO  9001  clauses.  Likewise,  ISO  9001  does  not  directly  cross-reference  with  the  CMMI-DEV, 
VI. 2  generic  practices,  or  even  specifically  to  generic  practices  4.1  and  4.2. 

For  all  the  other  process  areas,  one  could  say  that  ISO  9001  cross-references  to  the  generic 
practices  as  ISO  900 1  addresses  the  topics  of  the  generic  practices  on  a  one-time  (or  more)  basis 
for  the  individual  clauses  in  ISO  9001. 

The  following  table  shows  a  mid-level  comparison  of  the  content  relationships  between  ISO  900 1 
and  CMMI-DEV,  V 1 .2  maturity  level  2  process  areas. 

A  blue  cell  indicates  that  the  requirements  of  the  ISO  9001  standard  would  be  considered  to  be 
satisfied  by  CMMI-DEV,  V 1 .2  practices  without  any  additional  significant  guidance  or 
interpretation. 

A  green  cell  indicates  that  the  ISO  9001  clause  would  be  considered  to  be  satisfied  if  CMMI- 
DEV,  VI. 2  practices  were  fully  interpreted  and  implemented.  For  example,  clause  4.2.4  discusses 
the  requirement  for  quality  records,  yet  the  CMMI-DEV,  V 1 .2  model  does  not  have  any  directly 
equivalent  process  area  for  quality  record  requirements.  However,  generic  practice  3.2  of  CMMI- 
DEV,  V 1 .2  model  does  require  data,  infonnation,  and/or  measurements  to  be  retained  throughout 
the  implementation  of  the  model’s  practices. 

A  yellow  cell  indicates  that  the  requirements  of  the  ISO  9001  standard  may  be  satisfied  by 
CMMI-DEV,  V 1 .2  practices  if  significant  additional  interpretation  is  undertaken.  For  example,  up 
to  maturity  level  3,  there  is  no  direct  requirement  to  conduct  corrective  action  in  a  manner  that 
involves  root  cause  analysis  and,  therefore,  the  CMMI-DEV,  V 1 .2  model  requirements  would  be 
satisfied  with  simple  remedial  action.  However,  should  an  organization  do  some  form  of  root 
cause  analysis  on  issues — in  effect,  start  to  address  the  requirements  of  maturity  level  5  by 
implementing  the  process  area  Causal  Analysis  and  Resolution — then  this  would  satisfy  clause 
8.5.1  of  ISO  9001:2000. 

A  red  cell  indicates  the  determination  that  nothing  in  the  CMMI-DEV,  V 1 .2  model  could  cause 
the  ISO  9001  requirement  to  be  satisfied  effectively. 

A  grey  cell  indicates  that  these  clauses  would  be  satisfied  through  implementation  of  other  ISO 
9001  clauses  and  therefore  are  not  addressed  separately.  For  example,  clause  8.5.1,  which  requires 
continual  improvement,  is  actually  demonstrated  through  a  number  of  other  clauses:  quality 
policy  (5.3)  drives  objectives  (5.4.1),  which  are  checked  through  internal  audit  (8.2.2)  and 
analysis  of  data  (8.4),  leading  to  corrective  and  preventive  action  (8.5.2,  8.5.3),  which  is  reviewed 
by  management  (5.6).  Finally,  a  white  cell  indicates  that  there  is  no  significant  overlap  between 
the  CMMI-DEV,  VI. 2  model  and  the  ISO  9001  framework  in  these  particular  areas. 
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Table  6:  Mid-Level  Comparison  of  the  Content  Relationships  Between  ISO  9001  and  CMMI-DEV, 
VI. 2  Maturity  Level  2  Process  Areas 
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The  following  table  shows  a  mid-level  comparison  of  the  content  relationships  between  ISO  900 1 
and  CMMI-DEV,  V 1 .2  maturity  level  3  process  areas. 


Table  7:  Mid-Level  Comparison  of  the  Content  Relationships  Between  ISO  9001  and  CMMI-DEV , 
VI. 2  Maturity  Level  3  Process  Areas 
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Finally,  the  following  table  shows  a  mid-level  comparison  of  the  content  relationships  between 
ISO  900 1  and  the  CMMI-DEV,  V 1 .2  maturity  level  4  and  5  process  areas. 


Table  8:  Mid-Level  Comparison  of  Content  Relationships  Between  ISO  9001  and  CMMI-DEV,  VI. 2 
Maturity  Level  4  and  5  Process  Areas 
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4.3.5  T reatment  of  Customer  Satisfaction 


One  area  of  difference  that  a  number  of  previous  reports  have  noted  is  the  way  in  which  the  two 
bodies  of  knowledge  cover  customer  satisfaction. 

In  ISO  900 1 ,  the  treatment  is  explicit  and  immediate,  while  in  CMMI-DEV,  V 1 .2,  the  treatment  is 
indirect  and  implicit.  For  example,  in  ISO  9001,  clauses  5.2,  6.1,  8.2.1  and  8.4  all  make  explicit 
mention  of  customer  satisfaction.  In  CMMI-DEV,  V 1 .2,  the  term  customer  satisfaction  occurs  in 
only  three  minor  infonnative  references.  Note  that  this  does  not  mean  customer  satisfaction  is  not 
an  important  consideration  in  the  CMMI-DEV,  V 1 .2  model,  but  rather  is  more  of  a  reflection  of 
its  origins  and  audience.  Customer  satisfaction  is  embedded  indirectly  in  CMMI-DEV,  VI. 2  in  a 
number  of  ways,  most  especially  through  the  inclusion  of  the  definition  of  the  term  stakeholder 
and  its  pervasive  use  throughout  the  model,  as  well  as  in  the  Requirements  Development  and 
Validation  process  areas. 


4.3.6  Treatment  of  High  Maturity 

CMMI-DEV,  V 1 .2  is  quite  explicit  about  the  required  use  of  quantitative  methods  at  maturity 
levels  4  and  5,  although  even  at  maturity  level  2  there  are  explicit  requirements  for  a  measurement 
capability  across  the  organization. 

The  ISO  9001  family  requires  statistics  in  a  very  general  way,  and  ISO  9000  and  ISO  9004 
elaborate  on  statistics  and  recommend  self-assessments  to  a  model.  In  a  very  general  way,  the 
requirements  of  ISO  9000  and  ISO  9004  are  similar  to  those  of  the  CMMI-DEV,  VI. 2  model’s 
maturity  level  5  process  areas,  regarding  finding  the  root  causes  of  problems  and  making 
necessary  improvements. 

The  ISO  9000  glossary  defines  statistical  techniques  as  helping  to  measure,  describe,  analyze, 
interpret,  and  model  variability.  ISO/TR  10017  gives  guidance  on  statistical  techniques  in  a 
quality  management  system. 

ISO  9001  (clause  8.1)  requires  measurements  to  “...include  determination  of  applicable  methods, 
including  statistical  techniques,  and  the  extent  of  their  use.”  ISO  TR/10017  is  referenced;  its 
application  has  demonstrated  the  incorporation  of  high  maturity  concepts  in  ISO  9001  standards 
since  2003. 

In  ISO  9004,  section  8  (Measurement,  analysis  and  improvement),  section  8.1.2.j  recommends 
level  4  activity  that  is  consistent  with  maturity  level  4  in  CMMI-DEV,  VI .2:  “. . .the  use  of 
appropriate  statistical  or  other  techniques  can  help  in  the  understanding  of  both  process  and 
measurement  variation,  and  can  thereby  improve  process  and  product  performance  by  controlling 
variation...” 

ISO  9001  sets  the  very  general  basis  for  high  maturity  activities  with  clause  8.4,  which  requires 
that  the  analysis  of  data  shall  provide  information  relating  to  characteristics  and  trends  of 
processes  and  products  (generally  similar  to  the  CMMI-DEV,  V 1 .2  process  area  Organizational 
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Process  Performance)  and  including  opportunities  for  preventive  action  (generally  similar  to  the 
CMMI-DEV,  VI. 2  process  area  Quantitative  Project  Management). 

ISO  9004  clause  8.4  builds  on  the  trends  above  in  a  way  similar  to  CMMI-DEV,  VI. 2  maturity 
level  5  by  saying  that  “. .  .the  analysis  of  data  can  help  to  detennine  the  root  cause  of  existing  or 
potential  problems  (similar  to  CMMI-DEV,  VI. 2  process  area  Causal  Analysis  and  Resolution) 
and  therefore  guide  decisions  about  the  corrective  (similar  to  CMMI-DEV,  V 1 .2  process  areas 
Causal  Analysis  and  Resolution  &  Organizational  Innovation  and  Deployment)  and  preventive 
actions  needed  for  improvement  (similar  to  CMMI-DEV,  V 1 .2  process  area  Organizational 
Innovation  and  Deployment).” 

Finally,  ISO  9004  clause  8.4  states  that  analysis  of  data  addresses  decision  making  (related  to 
CMMI-DEV,  VI. 2  process  area  Decision  Analysis  and  Resolution):  “Decisions  based  on  facts 
require  effective  and  efficient  actions  such  as 

•  valid  analysis  methods 

•  appropriate  statistical  techniques 

•  making  decisions  and  taking  actions  based  on  results  of  logical  analyses,  as  balanced  with 
experience  and  intuition.” 

4.4  Appraisal 

This  area  is  probably  the  source  of  as  much  confusion  as  any  other,  due  to  many  similarities  and  a 
number  of  subtle  but  significant  differences.  The  above  discussion  on  terminology  differences  has 
already  revealed  that  the  results  of  appraisal  in  the  two  approaches  hold  fundamentally  different 
meanings. 

In  the  world  of  CMMI-DEV,  VI. 2,  an  organization  undergoes  a  fonnal  SCAMPI  A  appraisal  that 
results  in  either  a  maturity  level  rating,  a  set  of  process  capability  ratings,  or  both.  The  appraisal 
results  can  then  be  posted  on  the  SEI’s  published  appraisal  results  website  upon  request.  The  SEI 
does  not  operate  a  certification  scheme. 

In  the  ISO  900 1  world,  an  organization  undergoes  an  ISO  900 1  audit  that,  if  successful,  results  in 
an  accredited  certificate  indicating  conformance  to  ISO  9001  within  the  scope  of  the  organization 
and  audit,  based  on  the  samples  of  the  organization’s  processes  recorded  in  the  audit  report. 

An  accredited  certificate  is  one  that  has  been  issued  by  an  accredited  certification  body,  such  as 
Lloyd’s  Register  (LRQA),  the  British  Standards  Institution  (BSI),  or  Det  Norske  Veritas  (DNV). 
These  accredited  certification  bodies  are  themselves  regulated  and  audited  by  national 
accreditation  bodies,  such  as  the  UKAS  in  the  UK  or  ANAB  in  the  U.S.,  against  international 
standards.  The  accreditation  process  ensures  that  the  certification  bodies  apply  and  undertake  the 
certifications  assessments  in  an  appropriate  and  professional  manner.  For  example,  checks  are 
performed  to  ensure  that  the  certification  bodies  have  appropriate  processes  in  place;  that  the 
assessments  are  conducted  by  appropriately  trained,  competent,  and  certified  auditors;  and  that 
there  is  no  indication  of  parallel  consultancy  being  provided  by  the  certification  body.  To  achieve 
this  goal,  the  recognition  of  an  accredited  certificate  is  generally  considered  very  important. 
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The  recognition  of  an  accredited  certificate  is  generally  considered  essential  in  providing 
confidence  to  an  organization’s  customers.  Additionally,  a  certificate  awarded  by  an  accredited 
certification  body  will  carry  the  logo  of  the  accreditation  body. 

As  mentioned  in  clause  3,  ISO  9001  audit  results  are  recorded  in  the  final  audit  report  for 
registration,  and  findings  impacting  registration  are  recorded  as  non-conformances,  typically  as 
either  minor  or  major.  Minor  non-conformances  are  similar  to  SCAMPI’s  Largely  Implemented 
(LI)  characterizations,  while  major  non-conformances  are  similar  to  SCAMPI’s  Partially 
Implemented  (PI),  Not  Implemented  (NI),  or  Not  Yet  (NY)  characterizations. 

One  interesting  difference  from  the  SCAMPI  method  is  that  in  an  ISO  9001  audit,  there  are  no 
requirements  for  a  minimum  number  of  team  members  to  be  present  at  any  particular  interview 
session.  In  fact,  in  most  cases,  team  members  work  independently,  apart  from  regular  schedule 
audit  team  meetings  where  data  and  information  is  shared  between  the  team.  As  a  consequence, 
there  is  a  cost  differential  between  the  two  activities,  although  proponents  of  the  CMMI-DEV, 

V 1 .2  approach  would  argue  that  there  are  substantial  benefits  associated  with  the  use  of  teams 
comprised  of  practitioners  and  managers  from  the  organization  being  examined.  There  are  other 
differences  that  translate  to  the  bottom-line  costs  associated  with  ISO  versus  CMMI-DEV,  VI. 2; 
an  ISO  9001  Lead  Auditor  needs  only  one  week  of  formal  training,  while  the  CMMI-DEV,  V1.2 
SCAMPI  Lead  Appraiser  needs  three  sequential  formal  training  courses  and  specified  related 
experiences.  The  result  is  a  dramatic  cost  difference  that  is  passed  on  to  the  organization  being 
appraised.  Again,  the  costs  of  each  approach  need  to  be  weighed  against  the  business  value  that  is 
gained  from  implementing  the  approach. 

4.5  Training  of  Lead  Appraisers  and  Lead  Auditors 

SCAMPI  Lead  Appraisers  must  satisfy  formal  educational,  continuing  education,  and  experience 
requirements  as  a  prerequisite  to  entry  into  SCAMPI  Lead  Appraiser  training,  and  they  must  also 
be  observed  successfully  completing  a  SCAMPI  A  appraisal.  Once  authorized,  they  must  deliver 
SCAMPI  services  under  the  auspices  of  an  approved  SEI  Partner  organization  that  assumes 
responsibility  for  their  professional  conduct. 

Entry  into  SCAMPI  Lead  Appraiser  Training  requires  the  following: 

•  participation  as  an  appraisal  team  member  on  at  least  two  SCAMPI  A  appraisals  or  on  one 
SCAMPI  A  appraisal  and  two  SCAMPI  B  or  C  appraisals  within  the  prior  24  months 

•  at  least  ten  years  of  project  management  and  engineering  experience  in  systems  or  software 
engineering 

•  a  minimum  of  two  years  of  experience  managing  technical  personnel 

•  an  advanced  degree  in  a  related  technical  area  or  equivalent  experience 

•  successful  completion  of  the  three-day  Introduction  to  CMMI  course 

•  successful  completion  of  the  five-day  Intermediate  Concepts  of  CMMI  course  [CMMI 
2008a] 

•  successful  completion  of  the  five-day  SCAMPI  Lead  Appraiser  training 
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In  order  to  gain  the  additional  credential  as  a  high  maturity  SCAMPI  Lead  Appraiser,  appraisers’ 
applications  must  be  accepted  by  the  SEI,  and  the  appraisers  must  pass  a  written  examination. 

This  credential  enables  them  to  lead  appraisals  for  which  the  target  maturity  level  is  4  or  5. 

ISO  900 1  Lead  Auditor  candidates  must  also  undergo  appropriate  and  recognized  certified 
training  in  order  to  conduct  audits  that  would  result  in  an  accredited  certificate  for  an 
organization.  This  training  typically  consists  of  a  one-week  Lead  Auditor  course,  which 
culminates  in  an  open  book  examination.  The  training,  which  will  have  been  accredited  by  a 
national  accrediting  agency,  consists  of  lecture -based  material,  exercises,  role-playing  practice 
audits,  and  an  examination.  After  completing  the  Lead  Auditor  exam  successfully,  the  candidate 
becomes  a  Trainee  Auditor. 

To  advance,  the  candidate  must  complete  four  audits  of  a  Quality  System  covering  20  audit  days 
of  effort  under  the  guidance  of  a  Lead  Auditor.  Following  completion,  the  Certification  Body 
Management  will  review  this  performance;  if  the  review  is  favorable,  the  Trainee  Auditor  will  be 
upgraded  to  an  Auditor. 

After  the  Auditor  completes  three  audits  in  a  lead  role  covering  1 5  audit  days  of  effort,  his  or  her 
performance  is  reviewed;  if  performance  is  deemed  acceptable,  the  Auditor  is  upgraded  to  a  Lead 
Auditor. 

The  Auditors  are  also  qualified  in  certain  EAC  code  sectors  (industry  sectors)  based  on  their 
education  and  experience  in  that  sector.  The  performance  of  Auditors  and  Lead  Auditors  is 
periodically  monitored  by  the  Certification  Body  and  the  records  of  the  same  are  maintained.  The 
Accreditation  Body  audits  these  records  during  the  yearly  surveillance  audit. 

4.6  Supporting  infrastructure 

Both  approaches  have  significant  support  networks  in  place.  The  main  differences  are  that  the  SEI 
does  not  have  a  true  certification  scheme  currently  established  and  the  appraisal  bestows  one  of 
five  levels  of  maturity,  while  the  ISO  900 1  does  have  a  certification  scheme  with  a  single 
outcome — pass  or  fail. 

Additionally,  the  CMMI  initiative  parallels  ISO  9001,  in  the  sense  that  the  CMMI  initiative  is 
partially  DoD-  and  industry-sponsor-driven,  while  ISO  9001  is  driven  by  the  associated 
international  standardization  infrastructure. 

4.7  Adoption 

This  section  focuses  on  the  current  degree  of  adoption  of  the  two  approaches  by  examining  the 
number  of  organizations  that  have  achieved  formal  levels  of  achievement,  as  well  as 
miscellaneous  secondary  indicators  of  adoption.  The  intent  is  not  to  suggest  that  one  or  the  other 
approaches  is  “better”  or  “worse”  than  the  other  based  on  the  comparison  of  the  degrees  of 
adoption,  but  rather  to  simply  provide  some  factual  information  on  the  extent  of  the  adoption  of 
the  two  approaches. 

As  ISO  900 1  is  a  generic  standard  for  all  organizations,  it  has  a  much  larger  deployment  area  than 
the  CMMI  initiative.  Many  product  development  organizations  are  finding  advantages  for  using 
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ISO  9001  and  CMMI-DEV,  VI. 2  in  tandem.  ISO  9001  provides  confidence  to  the  casual 
customer  and  CMMI-DEV,  V 1 .2  provides  details  for  process  improvements. 
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4.8  Comparative  Summary 

Table  9:  Comparison  of  Attributes  of  ISO  9001  and  CMMI-DEV,  VI. 2 


Attribute 

ISO  9001 

Capability  Maturity  Model-Integrated  for 
Development  (CMMI-DEV) 

Applicability 

All  organizations 

Organizations  that  develop  products 

Source  of 

Requirements 

ISO  9001 :2000,  ISO 

9000:2000 

CMMI-DEV,  VI. 2 

Doc.  Size 

24,  36  pages 

600  pages 

Doc.  Cost 

$52  +  $52 

Free  download 

Cognizant  Body 

ISO  TC  176 

The  Software  Engineering  Institute  (SEI) 

(U.S.  DoD-sponsored) 

Organization 

SC  1, 2,  3 

Revisions,  Training,  Auditors,  Best  Practices 
Conf,  Trials 

Advising  Bodies 

National  (e.g.,  U.S.  TAG) 

SEI  Partners,  CMMI  Steering  Group,  SEI 
Partner  Advisory  Board 

Accreditation  Bodies 

Depends  on  the  country  of 
interest 

The  SEI  Is  probably  the  closest  thing  in  the 
CMMI  world. 

Support 

Depends  on  the  country  of 
interest 

The  Software  Engineering  Institute  (SEI) 

(U.S.  DoD-sponsored) 

Conformance 

Pass/Fail 

Organization  can  receive  a  capability  level 
and/or  a  maturity  level. 

Conformance 

ISO  19011, 

Standard  CMMI  Appraisal  Method  for 

Document 

Guide  62 

Process  Improvement  (SCAMPI)  for  class  A, 

B,  or  C 

Certified  Bodies  (CB) 

100’s 

One:  The  SEI 

Certified  Training  Org. 

20  in  U.S. 

The  SEI  and  other  SEI  Partners 

Qualified  Auditors 

Thousands 

1 00+  Lead  Appraisers 

Conformance 

780K  Certifications 

3000+  Appraisals 

Major  Issues 

Certification  Creditability 

Training  and  appraisal  costs 

Liaisons 

Sectors 

ISO/IEC  JTC1  SC7,  IEEE,  INCOSE,  NDIA 

Guidance  Books 

Many 

Many 
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Appendix  A:  Individual  and  Organizational  Contributors 


Sponsors 

The  following  organizations  contributed  as  sponsors  of  this  initial  report.  Sponsorship  took  the 
form  of  direct  financial  contributions,  program  management,  or  by  sanctioning  members  to 
contribute  to  the  development  of  the  report. 

•  Lockheed  Martin  Integrated  Systems  and  Solutions 

•  Software  Engineering  Institute 

•  Z1  Quality  Management  committee  of  the  American  Society  for  Quality 


Authors 

These  individuals  developed  the  report  and  contributed  new  or  existing  material: 

•  David  H.  Kitson  (Software  Engineering  Institute  Visiting  Scientist — project  manager  and 
report  editor,  High  Maturity  SCAMPI  Lead  Appraiser,  SCAMPI  Lead  Appraiser  Observer, 
Introduction  to  CMMI  V 1 .2  Instructor,  Intermediate  CMMI  V 1 .2  Instructor,  former  working 
group  convener  for  ISO/IEC  JTC1  WG22) 

•  Robert  Vickroy  (ABS  Quality  Evaluations,  Inc.,  SCAMPI  Lead  Appraiser,  Introduction  to 
CMMI  VI. 2  Instructor,  ASQ-CQA,  ICCP-CDP,  EDPA-ISACA  and  ISC2  security  auditor, 
NQA-1  auditor) 

•  John  Walz  (The  Sutton  Group,  Z1  Quality  Management  committee  representative) 

•  David  Wynn  (Computer  Sciences  Corporation,  SCAMPI  Lead  Appraiser,  Registered  ISO 
9001 :2000/Tick  IT  Lead  Auditor) 

I  wanted  to  give  special  mention  to  the  contributions  made  by  my  fellow  authors,  and  to  again 
thank  them  for  their  continued  support  and  patience.  Bob  Vickroy  was  especially  strong  in  his 
expertise  in  both  bodies  of  knowledge  and  made  more  valuable  suggestions  than  I  could  absorb. 
John  Walz  did  an  exemplary  job  of  serving  as  liaison  with  the  Z1  committee.  Dave  Wynn  gave 
freely  of  his  extensive  work  in  mapping  the  two  bodies  of  knowledge  and  was  a  great  source  of 
insight.  Thanks  also  go  to  the  Tick  IT  International  Journal  for  giving  permission  for  the  inclusion 
of  some  of  Dave’s  mapping  work  first  published  there  [Wynn  2005,  Wynn  2006,  Wynn  2007]. 


Reviewers 

These  individuals  reviewed  and  commented  on  the  draft  report  prior  to  publication.  Note  that 
some  of  the  reviewers  were  invited  to  be  authors,  but  for  varying  reasons  were  unable  to  commit 
to  that  role  at  the  time  a  commitment  was  needed: 
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•  L.L.  “Buddy”  Cressionnie  (Lockheed  Martin  Aeronautics,  Quality  &  Mission  Success 
Processes  Senior  Manager,  RABQSA  Aerospace  Industry  Experience  Auditor,  IRCA  QMS 
and  EMS  Lead  Auditor,  U.S.  Technical  Advisory  Group  (TAG)  to  ISO/TC  176  Voting 
Member,  Z1  Committee  member) 

•  Scott  Duncan  (ASQ  Software  Division,  TSP  Coach) 

•  Margaret  Glover  (Software  Engineering  Institute,  High  Maturity  SCAMPI  Lead  Appraiser, 
SCAMPI  Lead  Appraiser  Observer,  Introduction  to  CMMI  V 1 .2  Instructor,  Intermediate 
CMMI  V 1 .2  Instructor) 

•  Venkat  Gopalan  (Cyber  Solutions,  Inc.,  SCAMPI  Lead  Appraiser,  Introduction  to  CMMI 
V 1 .2  Instructor) 

•  Eugene  Kirsch  (Booz  |  Allen  |  Hamilton,  Z1  Committee  member) 

•  Mike  Konrad  (Software  Engineering  Institute,  Introduction  to  CMMI  V1.2  Instructor,  CMMI 
IT  Instructor,  Intermediate  CMMI  V 1 .2  Instructor) 

•  Larry  McCarthy  (Motorola  Software  Group,  High  Maturity  SCAMPI  Lead  Appraiser) 

•  Norm  Moreau  (Theseus  Professional  Services,  LLC,  ISO  lead  auditor) 

•  Boris  Mutafelija  (Systems  and  Software  Consortium,  Inc.,  SCAMPI  Lead  Appraiser, 
Introduction  to  CMMI  V 1 .2  Instructor) 

•  Geetha  Partha  (Quality  Point  Integrating  Systems  LLC,  SCAMPI  Lead  Appraiser, 
Introduction  to  CMMI  V 1 .2  Instructor) 

•  Ramachandran  Partha  (Quality  Point  Integrating  Systems  LLC,  SCAMPI  Lead  Appraiser) 

•  Terry  Rout  (Software  Quality  Institute,  SCAMPI  Lead  Appraiser,  Introduction  to  CMMI 
V 1 .2  Instructor,  CMMI  IT  Instructor,  Intermediate  CMMI  V 1 .2  Instructor) 

.  Pedro  Sousa  (Cobra  Automotive  Technologies  S.p.A.) 

•  Angela  Tuffley  (Software  Quality  Institute,  SCAMPI  Lead  Appraiser,  Introduction  to  CMMI 
V 1 .2  Instructor,  CMMI  IT  Instructor,  Intermediate  CMMI  V 1 .2  Instructor) 

•  Joan  Weszka  (Lockheed  Martin  Systems  &  Software  Resource  Center,  Corporate 
Engineering  and  Technology,  Introduction  to  CMMI  VI. 2  Instructor) 
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Appendix  B:  List  of  Acronyms 


ISO  Acronyms 

BSI — British  Standards  Institute 

ISO — International  Organization  for  Standardization 

SC — Sub-committee 

TC — Technical  Committee 

ISO  9001  Body  of  Knowledge  Acronyms 

AB — accreditation  body 
CB — certification  body 

CMMI  Body  of  Knowledge  Acronyms 

CMM — Capability  Maturity  Model 

CMMI — Capability  Maturity  Model  Integration 

CMMI-ACQ— CMMI  for  Acquisition 

CMMI-DEV— CMMI  for  Development 

CMMI-SVC— CMMI  for  Services 

CMMI  SG— CMMI  Steering  Group 

DOD — Department  of  Defense 

NDIA — National  Defense  Industrial  Association 

SEI — Software  Engineering  Institute 

PA — process  area 

SP — specific  practice 

GP — generic  practice 

SG — specific  goal 

GG — generic  goal 

SCAMPI — Standard  CMMI  Appraisal  Method  for  Process  Improvement 

ARC — Appraisal  Requirements  for  CMMI 

EPG — Engineering  Process  Group 

SEPG — Software  Engineering  Process  Group 
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Appendix  C:  Terminology  Differences 


Table  10:  A  Comparison  of  Terminology  Definitions  for  ISO  9000  and  CMMi-DEV,  VI. 2 


ISO  9000 

CMMI  for  Development,  VI. 2 

3.9.1  Audit — “systematic,  independent  and 
documented  process  for  obtaining  audit 
evidence  and  evaluating  it  objectively  to 
determine  the  extent  to  which  audit  criteria 
are  fulfilled” 

Audit — “in  CMMI  process  improvement  work,  an  objective 
examination  of  a  work  product  or  set  of  work  products 
against  specific  criteria  (e.g.,  requirements).”  Clearly,  this 
definition  refers  to  the  use  of  the  term  audit  as  an  activity 
taking  place  within  the  product  developing  organization. 

3.1 .5  Capability — “ability  of  an 
organization,  system  or  process  to  realize  a 
product  that  will  fulfill  the  requirements  for 
that  product” 

Capability  level — “achievement  of  process  improvement 
within  an  individual  process  area.  A  capability  level  is 
defined  by  the  appropriate  specific  and  generic  practices 
for  a  process  area” 

2.9  Continual  improvement — “The  aim  of 
continual  improvement  of  a  quality 
management  system  is  to  increase  the 
probability  of  enhancing  the  satisfaction  of 
customers  and  other  interested  parties.” 

Process  improvement — “a  program  of  activities  designed 
to  improve  the  performance  and  maturity  of  the 
organization’s  processes  and  the  results  of  such  a 
program” 

3.6.6  Corrective  action — “action  to 
eliminate  a  detected  nonconformity” 

Corrective  action — “acts  or  deeds  used  to  remedy  a 
situation,  remove  an  error,  or  adjust  a  condition” 

3.3.5  Customer — “organization  or  person 
that  receives  a  product” 

Customer — “the  party  (individual,  project,  or  organization) 
responsible  for  accepting  the  product  or  for  authorizing 
payment.  The  customer  is  external  to  the  project,  but  not 
necessarily  external  to  the  organization.  The  customer 
may  be  a  higher  level  project.  Customers  are  a  subset  of 
stakeholders.” 

3.6.3  Defect — “non-fulfillment  of  a 
requirement  related  to  an  intended  or 
specified  use” 

Defect  Density — “number  of  defects  per  unit  of  product 
size  (e.g.,  problem  reports  per  thousand  lines  of  code)” 

3.4.4  Design  and  development — “set  of 
processes  that  transforms  requirements  into 
specified  characteristics  or  into  the 
specification  of  a  product,  process,  or 
system” 

Development — “In  the  CMMI  Product  Suite,  not  only 
development  activities  but  also  maintenance  activities  may 
be  included.  Projects  that  benefit  from  the  best  practices 
of  CMMI  can  focus  on  development,  maintenance,  or 
both” 

3.7.1  Document — “information  and  its 
supporting  medium” 

Document — “a  collection  of  data,  regardless  of  the 
medium  in  which  it  is  recorded,  that  generally  has 
permanence  and  can  be  read  by  humans  or  machines. 

So,  documents  include  both  paper  and  electronic 
documents.” 

3.2.6  Management — “coordinated  activities 
to  direct  and  control  an  organization” 

Manager — “in  the  CMMI  Product  Suite,  a  person  who 
provides  technical  and  administrative  direction  and  control 
to  those  performing  tasks  or  activities  within  a  specified 
area  of  responsibility.  The  traditional  functions  of  a 
manager  include  planning,  organizing,  directing,  and 
controlling  work  within  an  area  of  responsibility.” 
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ISO  9000 

CMMI  for  Development,  VI. 2 

3.10.1  Measurement  management 
system — “set  of  interrelated  and  interacting 
elements  necessary  to  achieve  metrological 
confirmation  and  continual  control  of 
measurement  processes” 

Organization's  measurement  repository— “a  repository 
used  to  collect  and  make  available  measurement  data  on 
processes  and  work  products,  particularly  as  they  relate  to 
the  organization’s  set  of  standard  processes.  This 
repository  contains  or  references  actual  measurement 
data  and  related  information  needed  to  understand  and 
analyze  the  measurement  data.” 

3.8.1  Objective  evidence — “data 
supporting  the  existence  or  verity  of 
something” 

Objective  evidence — “as  used  in  CMMI  appraisal 
materials,  documents  or  interview  results  used  as 
indicators  of  the  implementation  or  institutionalization  of 
model  practices.  Sources  of  objective  evidence  can 
include  instruments,  presentations,  documents,  and 
interviews.” 

3.3.1  Organization — “group  of  people  and 
facilities  with  an  arrangement  of 
responsibilities,  authorities,  and 
relationships” 

Organization — “typically  an  administrative  structure  in 
which  people  collectively  manage  one  or  more  projects  as 
a  whole,  and  whose  projects  share  a  senior  manager  and 
operate  under  the  same  policies.  However,  the  word 
organization  as  used  throughout  CMMI  models  can  apply 
to  one  person  who  performs  a  function  in  a  small 
organization  that  might  be  performed  by  a  group  of  people 
in  a  large  organization.” 

3.4.1  Process— “set  of  interrelated  or 
interacting  activities  that  transforms  inputs 
into  outputs” 

Process — “activities  that  can  be  recognized  as 
implementations  of  practices  in  a  CMMI  model.  These 
activities  can  be  mapped  to  one  or  more  practices  in 

CMMI  process  areas  to  allow  a  model  to  be  useful  for 
process  improvement  and  process  appraisal.” 

3.4.2  Product — “result  of  a  process” 

Product — “The  word  “product”  is  used  throughout  the 

CMMI  Product  Suite  to  mean  any  tangible  output  or 
service  that  is  a  result  of  a  process  and  that  is  intended  for 
delivery  to  a  customer  or  end  user.  A  product  is  a  work 
product  that  is  delivered  to  the  customer.” 

3.4.3  Project — “unique  process,  consisting 
of  a  set  of  coordinated  and  controlled 
activities  with  start  and  finish  dates, 
undertaken  to  achieve  an  objective 
conforming  to  specific  requirements, 
including  the  constraints  of  time,  cost  and 
resources” 

Project — “a  managed  set  of  interrelated  resources  that 
delivers  one  or  more  products  to  a  customer  or  end  user. 
This  set  of  resources  has  a  definite  beginning  and  end 
and  typically  operates  according  to  a  plan.  Such  a  plan  is 
frequently  documented  and  specifies  the  product  to  be 
delivered  or  implemented,  the  resources  and  funds  to  be 
used,  the  work  to  be  done,  and  a  schedule  for  doing  the 
work.  A  project  can  be  composed  of  projects.” 

3.1 .1  Quality — “degree  to  which  a  set  of 
inherent  characteristics  fulfils  requirements” 

Quality — “the  ability  of  a  set  of  inherent  characteristics  of 
a  product,  product  component,  or  process  to  fulfill 
requirements  of  customers” 

3.2.1 1  Quality  assurance — “part  of  quality 
management  focused  on  providing 
confidence  that  quality  requirements  will  be 
fulfilled” 

Quality  Assurance — “a  planned  and  systematic  means 
for  assuring  management  that  the  defined  standards, 
practices,  procedures,  and  methods  of  the  process  are 
applied” 
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3.2.5  Quality  objective — “something 
sought,  or  aimed  for,  related  to  quality” 

Quality  and  Process-Performance  Objectives— 

“objectives  and  requirements  for  product  quality,  service 
quality,  and  process  performance.  Process  performance 
objectives  include  product  quality;  however,  to  emphasize 
the  importance  of  product  quality,  the  phrase  quality  and 
process-performance  objectives  is  used  in  the  CMMI 

Product  Suite  rather  than  just  process  performance 
objectives. 

3.2.3  Quality  Management  System 
(QMS) — “management  system  to  direct  and 
control  an  organization  with  regard  to 
quality” 

This  term  has  no  direct  analogue  in  CMMI-DEV,  VI  .2. 

3.2.4  Quality  policy — “overall  intentions 
and  direction  of  an  organization  related  to 
quality  as  formally  expressed  by  top 
management” 

Organizational  policy — “A  guiding  principle  typically 
established  by  senior  management  that  is  adopted  by  an 
organization  to  influence  and  determine  decisions.” 

3.1 .2  Requirement — “need  or  expectation 
that  is  stated,  generally  implied,  or 
obligatory” 

Requirement — “(1)  a  condition  or  capability  needed  by  a 
user  to  solve  a  problem  or  achieve  an  objective.  (2)  a 
condition  or  capability  that  must  be  met  or  possessed  by  a 
product  or  product  component  to  satisfy  a  contract, 
standard,  specification,  or  other  formally  imposed 
documents.  (3)  a  documented  representation  of  a 
condition  or  capability  as  in  (1 )  or  (2)” 

3.3.6  Supplier — “organization  or  person 
that  provides  a  product” 

Supplier — “(1)  an  entity  delivering  products  or  performing 
services  being  acquired.  (2)  an  individual,  partnership, 
company,  corporation,  association,  or  other  service  having 
an  agreement  (contract)  with  an  acquirer  for  the  design, 
development,  manufacture,  maintenance,  modification,  or 
supply  of  items  under  the  terms  of  an  agreement 
(contract).” 

3.8.3  Test — “determination  of  one  or  more 
characteristics  according  to  a  procedure” 

Test  procedure — “detailed  instructions  for  the  setup, 
execution,  and  evaluation  of  results  for  a  given  test” 

3.2.7  Top  management— “person  or  group 
of  people  who  directs  and  controls  an 
organization  at  the  highest  level” 

Senior  Manager — “in  the  CMMI  Product  Suite,  a 
management  role  at  a  high  enough  level  in  an 
organization  that  the  primary  focus  of  the  person  filling  the 
role  is  the  long-term  vitality  of  the  organization  rather  than 
short-term  project  and  contractual  concerns  and 
pressures.  A  senior  manager  has  authority  to  direct  the 
allocation  or  reallocation  of  resources  in  support  of 
organizational  process  improvement  effectiveness.  (See 
also  ‘higher  level  management.’)  A  senior  manager  can  be 
any  manager  who  satisfies  this  description,  including  the 
head  of  the  organization.  Synonyms  for  senior  manager 
include  executive  and  top-level  manager.  However,  to 
ensure  consistency  and  usability,  these  synonyms  are  not 
used  in  CMMI  models.” 

3.5.4  Traceability — “ability  to  trace  the 
history,  application  or  location  of  that  which 
is  under  consideration” 

Traceability — “a  discernible  association  among  two  or 
more  logical  entities  such  as  requirements,  system 
elements,  verifications,  or  tasks” 
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3.8.4  Verification — “confirmation,  through 
the  provision  of  objective  evidence,  that 
specified  requirements  have  been  fulfilled” 

Verification — “confirmation  that  work  products  properly 
reflect  the  requirements  specified  for  them.  In  other  words, 
verification  ensures  that  ‘you  built  it  right.’” 

3.8.5  Validation — “confirmation,  through 
the  provision  of  objective  evidence,  that  the 
requirements  for  a  specific  intended  use  or 
application  have  been  fulfilled” 

Validation — “confirmation  that  the  product,  as  provided 
(or  as  it  will  be  provided),  will  fulfill  its  intended  use.  In 
other  words,  validation  ensures  that  ‘“you  built  the  right 
thing.’” 
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Appendix  D:  Body  of  Knowledge  Resources 


The  purpose  of  the  material  in  this  appendix  is  to  identify  primary  sources  of  information  for 
parties  wishing  to  maintain  currency  in  the  ISO  9000  and  CMMI  bodies  of  knowledge.  Resources 
identified  are  focused  on  primary  sources  of  current  authoritative  information,  publications  and 
conferences. 


ISO  9000  Body  of  Knowledge 

ISO  Management  Systems — a  bi-monthly  publication  from  the  ISO  Central  Secretariat 

Quality  Digest — a  (free)  monthly  publication  for  quality  assurance  and  control  professionals, 
from  top-level  managers  to  those  on  the  shop  floor. 

Quality  Systems  Update — a  monthly  industry  journal  focusing  on  ISO  9000-related  matters. 

TCI 76  web  site  (http://www.tcl76.org/default.asp) — this  is  the  “home”  for  the  technical 
committee  within  ISO  that  is  responsible  for  the  9000  family  of  standards. 

ISO  900 1  Auditing  Practices  Group 

(http://isotc.iso.org/livelink/livelink/fetch/2000/2I22/138402/I38403/3541460/customview.html7f 
unc=ll&objId=3541460&objAction=browse&sort=name) — this  is  the  “home”  for  the  auditing 
practices  group 

Tick  IT  International — The  quarterly  journal  of  the  Tick  IT  software  quality  certification  scheme 
ISSN  1354-5884;  frequently  contains  articles  of  interest  to  both  the  ISO  9000  as  well  as  the 
CMMI  community,  http://www.tickit.org/intemational.htm 


CMMI  Body  of  Knowledge 

SEI  website  (http://www.sei.cmu.edu) — this  is  the  primary  source  for  information  and  status  for 
the  CMMI  Product  Suite. 

CMMI  User’s  Group  conference — an  annual  conference  focusing  on  CMMI-related  matters 
sponsored  by  the  NDIA  in  conjunction  with  the  SEI.  Annual  attendance  is  300-500. 

SEPG  Conferences —  annual  conferences  focusing  on  engineering  process  groups  and  their 
success;  annual  attendance  at  SEPG  North  America  is  from  1000-1500. 

Y ahoo  CMMI  discussion  group — an  informal  and  unofficial  discussion  group  that  CMMI 
practitioners  often  use  to  help  sort  out  complex  CMMI-related  matters. 

SCAMPI  MDD — resource  for  SCAMPI  Lead  Appraisers  to  provide  requirements,  activities,  and 
practices  associated  with  each  of  the  processes  that  comprise  the  SCAMPI  method. 
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Training  courses — provided  by  the  SEI  and  by  SEI  Partners.  A  full  list  of  courses  is  available  on 
the  SEI  website. 
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